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PREFACE 


What would a transformed higher education system look like? What is the contribution 
of higher education institutions to a country? How do we create institutions that are 
ready to welcome a diverse range of students? How do we create institutions that 
challenge our students to develop the next big idea and address 21 st century challenges 
while providing them with the support that enhances their chances of success? 


The Department of Higher Education and Training (DHET) supports the contention of 


the true meaning of transformation — when all students 
entering the system have a reasonable chance of success and 
access to powertul forms of knowledge and practices that will 
enable them to enter the productive economy and improve 
their life chances and that of their families (DHET 2016:92). 


During a student success conference in 2016, a vice-chancellor from a South African 
university observed that the higher education system has always had low success and 
graduation rates. The necessity to identity a key contributing factor to student success, 
which could be measured and used in conjunction with other data sources to guide 
interventions, put student engagement on the South African radar. 


The journey for the study of student engagement in South Africa started a decade ago 
with the first full institutional survey in 2007 at the University of the Free State (UFS) to 
change this track record. From its inception, the focus was on supporting and developing 
the talented students of South Africa through the use of evidence, and the project 
aimed to create deeply contextualised measures that were globally benchmarked. The 
intention of the original United States-based student engagement survey of countering 
an obsession with rankings by refocusing the quality of higher education debate on 
students and their learning resonated strongly with the South African team. 


In 2007, the concept of student engagement was not well known in South Africa, even 
though it was firmly established in the United States. Interest in this field of research 
came into national focus after the Council on Higher Education commissioned a 
national project on student engagement to better understand what it is that students 
do while they are at university and how this might impact on their success. This initial 
project ran from 2008-2009 and included 11 institutions. After a review of the student 
engagement measure internationally and nationally, the Kresge Foundation provided 
funding to take the project into its next level of development, which currently involves 
four surveys adapted for the South African context administered in 15 public higher 
education institutions, with a growing number of private higher education institutions 
participating and showing interest. 
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The project provides a data-driven depiction of student experiences, which could be 
used by institutional leaders, academics and support staff to create conditions that 
impact student development and success. In some cases, institutions have used the 
data to facilitate change, and in other cases operational demands and changing 
institutional priorities have overtaken the importance of the use of the data. 


Throughout the development of the project, being a student in South African higher 
education has not become easier, with increasing financial pressure, decreasing state 
expenditure and increased accountability on how state resources are spent and how 
they are contributing to the development of the country. The #MustFall campaign 
of 2015 and 2016 highlighted the need for a realignment of students and their 
institutions. We believe that knowing what and how students learn, and what they are 
able to do as a result of their higher education experience, are critical for the success 
of the North American as well as South African higher education systems. Student 
engagement data represent the student experience in our institutions and classrooms 
today. By using the data institutions empower students as agents of change and involve 
them in obtaining a quality education that will transform not only the institution but 
also their lives. 


PURPOSE OF THIS BOOK 

This publication has the dual aim of contributing a South African perspective to a 
rich history of conceptual and empirical research on student learning and success, 
while also showing how student engagement data form a bridge between academic 
research and professional practice (McCormick, Kinzie & Gonyea 2013). To achieve 
the dual aims, evidence from the South African surveys of student engagement and 
leading international experts have been integrated to: 

= introduce student engagement as a field and a key to success; 

= further a systemic understanding of the student experience; 

=a illustrate how entering students can be better understood and supported; 

= facilitate the development of academics through student engagement; 

= stimulate the development of South African high-impact practices; 


= illustrate how student engagement evidence can be leveraged for institutional 
change and improvement; 


= explore the role of student affairs in student engagement and learning; 
= highlight the role of institutional leadership in advancing student engagement; 
= promote pedagogical practices that matter for student success; and 


= demonstrate how we are using engagement data for change and empowerment 
at course level. 


We close by taking stock of student engagement in South African higher education and 
sharing the plan moving forward with the variety of stakeholders who are committed 
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to improving student success and the effectiveness and efficiency of higher education. 
They include governmental, public and private institutional stakeholders, as well as 
intra-institutional faculties, departments and individuals. The content will also be of 
interest to both researchers in higher education as well as leaders and practitioners 
who would like to integrate evidence into their institutional roles. 


HOW TO READ AND USE THIS BOOK 


The book consists of three parts, complemented by an introduction (Chapter 1) 
as well as the summary chapter (Chapter 11) that includes some reflections and 
recommendations. 


Part | provides a status report on student engagement in South Africa. Chapter 2 
shares data from the South African Survey of Student Engagement (SASSE) to provide 
a systemic perspective on the experiences of students in higher education. Chapter 3 
shares data from the Beginning University Survey of Student Engagement (BUSSE) 
on the expectations of first-year students and contextualises the data using the 
financial stress scale introduced in SASSE 2016. The next chapter explores lecturers’ 
perspectives on student engagement using data from the Lecturer Survey of Student 
Engagement (LSSE). The chapter considers similarities and differences between student 
and staff perspectives and discusses the implications for staff development. This section 
of the book closes with Chapter 5, which introduces the development of high-impact 
practices in the South African context, and shares SASSE data on the prevalence of 
these practices. 


Part II of the book presents perspectives about how to create institutional conditions 
that matter for student success. Chapter 6 shares strategies for leading evidence-based 
improvement through student engagement by using different ‘roles’ or designations as 
an international vehicle for engaging people in evidence-based change. Chapter 7 
addresses the role engagement data can play in assisting student affairs to advance 
student learning. The critical importance of learning and involving managers at all 
levels in promoting student engagement, as well as practical suggestions on how to 
do this are explored in Chapter 8. 


Part III shifts the focus to teaching and learning and how conditions for success can 
be created in the classroom. Chapter 9 explores the potential impact of effective or 
high-impact pedagogical practices on student success. Chapter 10 demonstrates 
how student engagement data from the Classroom Survey of Student Engagement 
(CLASSE) can be used to promote evidence-based improvement in pedagogy at the 
course/module level as well as evidence-based academic staff development. 


NOTES ON TERMINOLOGY 


Because this book has been authored by South African and United States scholars, 
some terms or concepts might need clarification: 
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In pre-democratic South Africa, racial groups were divided into “Indian”, “black”, 
“coloured” and “white”. To track transformation and equity interventions, these 
groupings are still used in research circles. However, during the contextualisation 
process to adapt the engagement surveys to the South African contexts, students 
were given a choice which racial classification categories they would prefer to 
have in the surveys. The options they selected include “black African”, “coloured”, 
“Indian”, “white”, “other”, “mixed-race”, and “l prefer not to answer”. Analyses 
throughout the book primarily rely on comparisons between black African and 
white students. This is because samples representing the other groups are small 
in comparison to black African and white groups who participated in the surveys. 


The term “first-generation” refers to whether students are the first in their immediate 
families to graduate from university. 


nou 


US references to “colleges”, “courses”, “faculty” (academic staff) have for the most 
part been changed to jargon more familiar to a South African audience, however, 
some references to these terms have been left unchanged to preserve the intended 
meaning the authors tried to convey. 


In the United States, reference to “sample” indicates the whole population from 
which respondents were selected/volunteered. However, when using the word 
“sample” in this text it refers to the respondents participating in the survey, and not 
the broader population which the sample forms part of. 
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STUDENT ENGAGEMENT: 
A KEY TO SUCCESS 


Francois Strydom 


May your choices reflect your hopes, not your fears — Nelson Mandela 


INTRODUCTION 


Expanding access to higher education and increasing students’ chances of success 
are two of the critical challenges facing global higher education. Of the many 
access challenges, affordability of higher education has taken centre stage in many 
countries, with large-scale protests taking place in South Africa. The protests are 
complicated by the fact that higher education researchers acknowledge that students 
are among the groups least informed about higher education (choices, outcomes 
and processes) (Mahat & Goedegebuure 2014). Therefore, it is vital that higher 
education institutions understand how students think, behave and learn as well 
as what they are able to do after higher education (Coates & McCormick 2014; 
Kuh et al 2015). 


This book aims to show how student engagement data can further an evidence- 
based understanding of the time and effort students expend in using the resources 
that institutions provide for their learning and personal development and how 
institutions can use student engagement data to improve conditions that support 
student success. The chapters provide university leaders, academics, support staff 
and policy makers with insights on how to promote student engagement. Focusing on 
student engagement promotes student agency by bringing a data-informed student 
perspective to institutional planning and higher education policy development. In 
addition, student engagement data allow institutions to promote effective educational 
practices aimed at enhancing the chances of success for all students and reducing 


the inequalities in our societies. 
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THE CASE FOR STUDENT ENGAGEMENT 


This section offers some reasons for promoting student engagement at all levels. 


Student engagement facilitates success 


Higher education research indicates that some of the best predictors of whether or 
not a student will graduate are academic preparation and motivation (Pascarella & 
Terenzini 2005). Unfortunately, the only possible way to control these two variables 
is to employ more stringent admission and/or selection policies. This is, however, not 
a viable alternative in a century where, internationally, higher education institutions 
are obligated to enrol more students from increasingly diverse backgrounds to meet 
the skills needs of the knowledge economy. Nor is it a fair alternative in highly 
unequal societies such as South Africa, where access to university could have life- 
changing consequences for those who have been excluded from such opportunities 
in the past. Years of research into effective higher education institutions in the United 
States point to a third factor that, at least marginally, can enhance the prospect that 
students will survive and thrive after entering higher education. Several decades 
of evidence suggest that after controlling for student background characteristics, 
student engagement (i.e. students devoting their time to educationally purposeful 
activities) is also a significant predictor of their satisfaction and success (Kuh, Kinzie, 
Schuh & Whitt 2010; Mayhew, Seifert, Pascarella, Nelson Laird & Blaich 2012). 
Student engagement also benefits underprepared students to a greater extent, thereby 


enhancing their chances of success (Kuh, Cruce, Shoup, Kinzie & Gonyea 2008). 


Student engagement research and practice is a global interest 


Figure 1.1 illustrates how focusing on student engagement has spread across the globe. 


The major source of engagement measures come from the National Survey of Student 
Engagement (NSSE), which has been contextualised in different countries. The dark 
shading represents established and cross-institutional data collection. The lighter 
shading indicates a one-time trail of emergent implementation that involve a smaller 
number or in some cases individual institutions (Coates & McCormick 2014). The 
broad use of this assessment tool has facilitated the development of an international 
network and has also created a bridge between conceptual and empirical research 
on student learning and development, as well as practical demands of assessment 
and improvement to enhance the quality of undergraduate education (McCormick, 
Kinzie & Gonyea 2013). 
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FIGURE 1.1 Mapping student engagement across the globe 


Student engagement provides reflection and accountability on education processes 


For better or worse, rankings are firmly ensconced in perceptions of the quality of 
higher education across the globe. The shortcomings of these rankings have been 
laid bare from philosophical and methodological perspectives (Gladwell 2011); 
a common thread in these critiques is that ranking emphasises reputation and 
input measures rather than teaching and learning (Kuh 2001, 2003). The NSSE 
was developed in response to ranking in order to intentionally focus on process 
indicators, or ‘good practices’ that prior research has related to desirable student 
outcomes (McCormick et al 2013). Student engagement measures provide data 
from the field in the form of an actionable evaluation of the institutional environment 
and the quality of student learning, which can be used to address accountability 
pressures and assess educational quality at institutional level and concerns about 
student success while contributing to the development of the scholarship of teaching 
and learning (McCormick et al 2013). In the South African context, the South African 
Survey of Student Engagement (SASSE) has been cited as a tool that can be used to 
enhance the quality of teaching and learning (Council on Higher Education [CHE] 
2015, 2016). 


DEFINING STUDENT ENGAGEMENT 


Student engagement is defined in terms of two key components. The first is the 
amount of time and effort students spend on academic activities and other 
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activities that lead to the experiences and outcomes that constitute student success. 
The second is the way that institutions allocate resources and organise learning 
opportunities and services to induce students to participate in and benefit from such 
activities (Kuh et al 2010). Put differently, student engagement can be defined by 
two key components: first, what students do (the time and energy they devote to 
educationally purposeful activities), and second, what institutions do (the extent to 
which they employ effective educational practices to induce students to do the right 
things academically) (Kuh 2001). 


THEORETICAL AND EMPIRICAL FOUNDATION OF STUDENT ENGAGEMENT 


What differentiates student engagement measures from other measures is its laser- 
like focus on activities and experiences that have been empirically linked to desired 
outcomes in higher education (NSSE 2000). Student engagement spans the fields of 
psychology, sociology, cognitive development and learning theory, and is linked with 
a long tradition of impact research (McCormick et al 2013). The roots of student 
engagement can be traced back to the educational research that emerged as early 
as the 1930s with Tyler’s focus on the importance of the amount of time spent on 
academic tasks (Merwin 1969) and later longitudinal research by Pace (1980, 1984, 
1998) into the effect of quality of effort on desired student outcomes (Kuh 2009) 
underlining the importance of student agency. This is shown by the following quote: 

We have typically thought of educational processes in terms of what 

they contribute to the product; but we know that some processes are 

qualitatively better than others, just as some products are better than 

others, so perhaps we should give more thought to measuring the quality 

of the processes. One motivation for my desire to measure student effort 

was the recurring rhetoric about accountability that always blamed the 

institution for outcomes ... This assumes that the student is buying a 

product when actually the student, at a later point in time, is the product. 

So, the other side of accountability is the quality of effort students 


invest in using the facilities and opportunities the college provides 
(Pace 1998:28). 


Over the course of more than 30 years, Astin’s research on student involvement 
(1984, 1977, 1993, 1999) has confirmed that any form of student involvement 
(the amount of physical and psychological time and energy the student invests in 
the educational process) is positively associated with a wide variety of academic 
outcomes. In fact, one of the primary findings in How College Affects Students 
(Mayhew et al 2016), is that the time and energy students devote to their own 
educational experience relates directly to student success. Student engagement 
scholarship places a strong emphasis on student involvement in learning in terms of 
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the quality of effort, as well as time spent on tasks. It is important to mention that in 
addition to pointing to behaviours, students need to engage in both. Astin and Pace 
both emphasised that institutions play an important role in creating environments 
that enable students to induce these behaviours (McCormick et al 2013). Vincent 
Tinto’s theory on academic and social integration was one of the first theories to 
emphasise that both the student and the institution had a role to play in keeping 
students from early and voluntary departure from higher education (Tinto 1986). In 
short, Tinto highlighted the role positive academic or social experiences could play 
in creating a sense of belonging within institutions, while negative experiences in 
these areas might lead to a sense of disconnect or isolation for the student. 


However, Kuh (2009) explains that student engagement extends beyond students’ time 
and involvement in their studies towards examining the extent of student participation 
in effective educational practices, as outlined by Chickering and Gamson (1987) in 
their landmark publication Seven Principles for Good Practices in Undergraduate 
Education. In this publication, Chickering and Gamson present seven principles that 
encompass what students should be doing during their undergraduate education to 
optimise their personal development and to promote effective learning. The primary 
premise of their work is that when students and staff take the responsibility to devote 
time and effort to tasks related to these principles, student learning and success 
will improve (Kuh & Vesper 1997). The principles are: student-staff interaction, 
cooperation between students, active learning, prompt feedback, time on task, 
high expectations of students, and lastly, respect for diverse talents and ways of 
learning. These widely researched principles continue to influence teaching and 
learning practices globally (Kuh & Vesper 1997) and recently published national 
longitudinal projects in the United States have confirmed the continued importance 
of implementing them (Blaich & Wise 2011). 


Recent developments in higher education highlight the importance of ‘high-impact 
practices’. Involvement in these practices or activities has been shown to advance 
capabilities in areas such critical thinking, solving real-world problems and working 
effectively with others regardless of the background from which students come. 
Examples of these activities in the United States context include community service 
learning, being part of learning communities, undergraduate research, internships, 
capstone/culminating projects, and study abroad opportunities (Kuh 2008). These 
activities are explored in greater detail in Chapter 5. 
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Having reflected on the theoretical and empirical foundation of student engagement, 
our focus moves to how student engagement facilitates the development of a 


conceptual framework to put together the puzzle of student success. 


PUTTING TOGETHER THE STUDENT SUCCESS PUZZLE 


Figure 1.2 presents a framework for understanding student success as a wide path 
or perhaps a direct path in which students work, but where there can be many 
twists, turns, detours, roundabouts and occasional dead-ends (Kuh, Kinzie, Buckley, 
Bridges & Hayek 2007:10), instead of the usual pipeline understanding of students 
entering and exiting education systems. Students can therefore enter at a specific 
time but exit, due to financial pressure or employment opportunities and return later 
to study further. 


FIGURE 1.2 Contextualised student success framework 


The arrow (left-bottom corner) summarises some of the many _pre-university 
experiences students enter higher education with, such as family background, 
academic preparation, attitudes to university readiness, family and peer support, and 


motivation to learn. Within the South African context, addressing the low levels of 
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language and numerical competence of learners exiting the secondary school system 
is a critical challenge leading to proposals for flexible curricula and programme 


structures, such as the four-year degree, to allow for innovative solutions. 


Some of the issues related to access to higher education include affordability and 
sustained funding, entry requirements and challenges associated with the transition 
into higher education. In South Africa, responses to these issues or challenges have 
been bursaries or loans, National Student Financial Aid Scheme (NSFAS) support, 
and now the Ikusasa Student Financial Aid Programme (ISFAP), as well as alternative 
access programmes. Also in the South African context, alternative access routes such 
as bridging and foundation programmes (extended degrees) as well as Recognition 
of Prior Learning (RPL) have helped to broaden and mediate entry into higher 
education. If learners are able to navigate these transitions successfully, they enter 


the ‘traditional’ higher education environment. 


The next part of the educational journey consists of a student’s university experience, 
namely: student behaviours and institutional conditions. Student behaviours include 
study habits, peer involvement, interaction with staff, time on task and motivation. 
Institutional conditions include resources, educational policies, programmes, 
practices and structural features. Student engagement, at the intersection of these 
behaviours and conditions, represents aspects of student behaviour and institutional 
conditions that universities have influence over, at least marginally. All the factors 
are intertwined and affect what students do during their time at university. Research 
into student retention shows the need for creating a more supportive mainstream 
environment for students rather than through access programmes, which underlines 
the importance of a focus on student engagement (Letseka, Cosser, Breier & 


Visser 2009). 


The arrows at the top-right hand corner represent successful student progress and 
the outcomes that higher education aims to deliver, namely, employable graduates, 
professional and/or postgraduate study and graduate attributes such as leadership 
and citizenship. By focusing on student engagement and using student engagement 
data, institutions can assess the prevalence of student behaviours and institutional 
conditions related to success use the data to develop interventions that can channel 


student energy to activities that matter to their success. 


Having reflected on the importance of student engagement within the success 
puzzle, the next section provides a more in-depth discussion on the measurement of 


student engagement. 
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MEASURING STUDENT ENGAGEMENT 
Since student engagement incorporates behaviours and perceptions, the ‘measures’ 
items gauge both these dimensions. Figure 1.3 shows how they are manifested in 


the specific measure. 


Behavioural manifestations 
Time on task 
Reading and writing 


Conceptual elements of Class participation and presentations 


student engagement Group work 


Gueliy ei cieri Higher-order cognitive tasks in courses 


Interaction with staff 
Involvement 


: Sie f Participation in events and activities 
Academic and social integration a ; 
PE oe High-impact practices 
Principles for good practice in 


undergraduate education Perceptual manifestations 


Quality of relationships 


High staff expectations 


Environmental support 


FIGURE 1.3 Conceptual elements of student engagement and selected manifestations 


The strong focus on measuring student behaviour differentiates the student 
engagement survey from other student experience measures, which tend to focus 
on values, attitudes or student satisfaction. The focus on behaviour allows for 
institutions to develop concrete and actionable interventions to target and change 
students’ behaviour. Additional strengths of the student engagement measures, 
which also promote quality research, include: the standardisation of survey sampling 
and administration to facilitate comparability between institutions; and providing 
participating institutions with comprehensive reports detailing their own students’ 
responses relative to those at comparison institutions, plus an identified student data 


file to permit further analysis by the institution. 


Developing the South African Survey of Student Engagement 


The piloting and development of the first version of the South African Survey of Student 
Engagement (SASSE) took place in 2006. The first full administration took place 
during 2007 with 752 students responding (Strydom, Kuh & Mentz 2010). The initial 


contextualisation of the measure included only slight adaptations from the NSSE. 
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This was to ensure that the vocabulary used was applicable to the local context. For 
example, words such as “college” were changed to “university”; “faculty” changed 
to “lecturer”, and so on. The original NSSE instrument is only available in English 
and to optimise its use in the South African context it was translated into Afrikaans. 
Back-translation was done to ensure the content validity of the Afrikaans version. 
Afrikaans was selected as an alternative language for this instrument since it is a 
formal language of instruction at several other South African universities, including 
the University of the Free State (UFS), where the administration of the South African 


engagement surveys takes place. 
Two of the items in the NSSE were changed entirely in the SASSE 1.0. They were: 


1. “Which of the following have you done, or plan to do before leaving your 
institution? Participate in a learning community or some other formal programme 


where groups of students take two or more classes together.” 
This item was changed to: 


“Which of the following have you done, or plan to do before leaving your 
institution? Participate in academic student societies (law, psychology, and so 


on) where students engage on topics related to their subject.” 


This change was proposed due to the more structured nature of degree 
programmes in South Africa. The proposed item attempts to measure student 


involvement in enriching academic activities. 


2. “Which of the following have you done, or plan to do before leaving your 
institution? Culminating senior experience (capstone course, senior project or 
thesis, comprehensive exam, and so on).” This item was changed to “Which 
of the following have you done, or plan to do before leaving your institution? 
Develop a community project in which you use your university knowledge to 


address a problem in your community.” 


The rationale for this change was that the South African higher education 
environment does not have culminating senior experiences. A community project 
would provide the student with an integration and application opportunity and 
would also align teaching and learning with the important South African higher 


education policy objective of community engagement (Strydom et al 2010). 


As with all self-report measures, the questions or items in these measures must be 
formulated in a way that enables the participants to respond to the survey designers’ 
intent. Based on research into self-report data — and used as a guideline to track 
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validity of the contextualised surveys — researchers identified the following conditions 
that are conducive to the validity of self-report measures: 


e The information requested is known to the respondents. 
e The questions are phrased clearly and unambiguously. 
e The questions refer to recent activities. 


e The respondents think the questions merit a serious and thoughtful 
response. 


e Answering the questions does not threaten, embarrass, or violate the 
privacy of the respondent or encourage the respondent to respond 
in socially desirable ways (Kuh, Hayek, Carini, Ouimet & Kennedy 
2001:9). 


SASSE 2.0: Deeply contextualised and globally benchmarked 


Following a national study sponsored by the Council on Higher Education in 2009- 
2010, the SASSE instruments were subjected to an extensive review. This was to: 


= ensure continued contextual relevance of the measure for the improvement of 
teaching and learning; 

= maintain alignment with international review of measures; 

= refine problematic items; 

= enhance the quality of data collection, analysis and reporting processes; 

= maximise the collection of ‘useful’ data in the South African higher education 
context; and 


= broaden our knowledge and understanding of the student experience in the 
South African context. 


The SASSE review process included inputs from institutional representatives who 
participated in the study and analysis of data from national administrators of 
the survey. It incorporated experiences of the research team that worked on the 
project, as well as the inputs of eight South African higher education experts and 
an international expert. To facilitate deep contextualisation, qualitative research was 
conducted at five institutions, in four provinces. In total, 68 students participated in 
individual interviews and 80 students in focus groups. 


The review processes resulted in sets of new, updated and continuing items. These 
were rigorously tested and grouped within several Engagement Indicators, organised 
within themes adapted from the former Benchmarks of Effective Educational Practice. 
Therefore, indicators focused on important aspects of educational quality, making 
them specific and actionable. 
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RANGE OF UPDATED STUDENT ENGAGEMENT SURVEYS AVAILABLE IN 
SOUTH AFRICA 

A range of student engagement surveys has been developed and contextualised for 
use in South Africa since 2007. These instruments enable institutions to monitor and 
develop capacity at an institutional, as well as individual course or module level, as 


shown in Figure 1.4. 


Institutional-level measures 


Beginning University Survey of Student Engagement (BUSSE) 


The BUSSE measures entering first-year students’ pre-university 
academic and curricular experiences and their expectations regarding 
participation in educationally purposeful activities during their first year 
at a tertiary institution. 


South African Survey of Student Engagement (SASSE) 


The SASSE gathers comprehensive information relating to the extent of 
student participation in effective educational practices as part of the 
teaching and learning experience. 


Lecturer Survey of Student Engagement (LSSE) 


The LSSE measures lecturer expectations regarding student engagement 
in educational practices that are empirically linked with high levels of 
learning development. 


Modular/Course-level measure 


Classroom Survey of Student Engagement (CLASSE) 


The CLASSE measure was developed to provide institutions with 

a diagnostic tool with which they are able to intervene in courses 
with high dropout and failure rates and creates an evidence-based 
approach to academic staff development. 


e CLASSE, sare asks the lecturer of that module/course how important 
the various educational practices are for facilitating student success. 


e CLASSE,, „gen asks students how frequently they engage in various 
educational practices within a specific course. 


€ 


FIGURE 1.4 Contextualised student engagement surveys available in South Africa 


BENEFITS OF USING STUDENT ENGAGEMENT SURVEYS AND DATA 
This section explores what institutions can learn from student engagement surveys 
and how the data can be used to empower students, academics, support staff 


and management. 
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Facilitates better understanding of contemporary students 


Understanding students has become essential to 21st century higher education. 
Kuh et al (2015) point out that understanding what students know and are able 
to do are fundamental to student success and to the quality and effectiveness of 
higher education. In South Africa, student protests have emphasised the need for 
more research to understand students. With vital contributions from national and 
institutional student engagement data, we have been able to identify areas where we 


need to understand students better. These include: 


= The need for greater academic differentiation between first-year and senior students. 

= Determining appropriate levels of academic challenge. 

= Understanding the financial stress students are under. 

= Understanding student expectations. 

*  Scrutinising how students learn, use support, and are taught. 

# Identifying and implementing high-impact practices for co-curricular learning and 
development. 


= |mpacting academic staff development and pedagogical practices through data- 
driven conversations (Strydom, Oosthuysen, Hen-Boisen, Henn & Posthumus 2015). 


Provides actionable data for improving student retention and progress 


More than a decade of research shows that in the United States student engagement 
“has significant positive, though modest, relationships with grades and persistence 
for students from different racial and ethnic backgrounds, even after controlling 
for a wide range of key precollege variables” (McCormick et al 2013:65). These 
positive effects are stronger for first-year grades and persistence to the second year 


for underprepared and historically underserved students (Kuh et al 2008). 


In the case of senior students, increased participation in effective educational 
practices exercises a small yet positive impact on the academic performance, while 
higher levels of engagement in the early years of college have a compounding 
effect on students’ grades at a later stage of their higher education experience 


(Kuh et al 2007). 


In the South African context, studies have started to find a positive relationship between 
academic achievement and student engagement. Swart (2014) found a positive 
relationship between 22 items associated with high-impact pedagogical practices and 
academic achievement amongst commerce students. Schreiber and Yu (2016) found 


that student engagement patterns are reliable predictors of academic performance. 
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Helps to identify constraints in the teaching and learning environment 


Student engagement surveys provide student self-report data on the extent to which 
students engage in educationally effective practices and the extent to which the 
institution makes use of these practices. Analysis of the data provides an institution with 
information about the extent to which students are engaged in effective educational 
practices such as studying, reading, tutoring, asking questions in class, to name a 
few. The same student engagement data set could be used to provide feedback 
on the extent to which students experience the institution as supportive, and their 
classroom as providing active and collaborative learning opportunities. Therefore, 
student engagement data can be used to develop interventions at either the student 
or staff levels to address practices that might constrain teaching and learning. When 
combined with data from the Lecturer Survey of Student Engagement (LSSE), the data 
provide institutions with a student and staff perspective on engagement as well as 
information on how staff say they are spend their time. This helps to provide a richer 
picture of possible constraints to good teaching and learning at the institutional level 
and across the higher education sector. Chapter 4 explores the staff perspective in 
greater detail. 


Facilitates the creation of high-impact curricula 


As mentioned earlier, the identification of high-impact practices is one of the more 
recent empirical contributions from the field of higher education. Examples of these 
activities in the United States context include community service learning, being part 
of learning communities, undergraduate research, internships, capstone/culminating 
projects, and study abroad opportunities (Kuh 2008). The 2007 Annual NSSE report, 
Experiences that Matter: Enhancing Student Success and Learning, documented a 
number of outcomes associated with participation in these high-impact practices. 
Amongst other things, the NSSE found that: 


= Learning communities are significantly associated with numerous personal 
development and learning gains. 


= Students conducting research with staff members are more likely to persist, gain 
more intellectually as well as personally, and choose a research-related field as a 
career. These students also participate more frequently in deep learning approaches 
and report greater personal growth and learning. 


= Study abroad was moderately related to deep learning and self-reported gains, 
even after controlling for various institutional and biographical characteristics. 
= Students reported that their culminating experience contributed substantially to their 


abilities in a number of areas, depending on the type of culminating experience 
(NSSE 2007). 
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At a time when there is increased focus on curriculum in the South African higher 
education the integration of high-impact practices offers a way to transform how 
undergraduate students are educated. Chapter 5 explores these practices in 
greater detail. 


Promotes evidence-based pedagogical innovation 

In addition to the identification of high-impact practices, the analyses of specific 

items the NSSE related to classroom teaching and learning practice have been 

found to have a positive impact on retention and persistence (Kinzie, Gonyea, 

Shoup & Kuh 2008). Specific items have been identified as high-impact pedagogical 

practices based on an analysis of their impact on students’ academic achievement 

and perseverance. The 19 pedagogical practices include items such as: 

= Asked questions in class or contributed to class discussions. 

= Made a class presentation. 

= Prepared two or more drafts of a paper or assignment before turning it in. 

= Worked with other students on projects during class. 

= Worked with classmates outside of class to prepare class assignments. 

= Participated in a community-based project as part of a regular course. 

= Received prompt feedback from faculty on your academic performance (written 
or oral). 

= Worked harder than you thought you could to meet an instructor’s standards or 
expectations. 

= Had serious conversations with students of a different race or ethnicity than 


your own. 


= Had serious conversations with students who differ from you in terms of their 
religious beliefs, political opinions, or personal values. 


By monitoring the use of these teaching and learning practices institutions can 
enhance the quality of teaching and learning and create more engaging learning 
experiences for students. Chapter 9 considers the 19 items identified in the United 
States context and performs a broader analysis in an attempt to identity practices that 
will contribute to retention and success in the local South African context. 


Enables systematic approaches to quality enhancement 
The NSSE and other surveys of student engagement were developed in part in reaction 
to the media ranking systems of higher education institutions in the United States. The 


aim with these surveys was to refocus the discussion of quality in higher education on 
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students and their learning (NSSE 2009). Similarly Coates (2005) indicates that in 
the Australasian context, student engagement data have the potential to strengthen 
quality assurance systems in higher education. He criticises the use of progress 
rates (success rates) for assessing quality on two levels. First, progress rates are an 
inadequate indicator of the student’s perception of institutional quality, and secondly, 
progress rates do not provide an objective measure of quality as they are relative 
to courses and/or modules. Coates maintains that high progress rates through a 
system might in fact be an indication of the lowering of academic standards. He 
suggests that student engagement focuses the discussion of institutional quality on 
student learning (an essential, if not the most important aspect of education) instead 
of the quality debate being monopolised by resources and institutional reputations 


(Coates 2005). 


McCormick (2009) expands on the value of student engagement surveys for 
the systematic monitoring of quality by indicating that the use of these measures 
promotes critical, internal self-reflection or reflective accountability. Because 
student engagement measures provide institutions with information about aspects 
that institutions can do something about, systematically gathering data on student 
engagement allows institutions to develop interventions within the parameters of 


their own mission. 


McCormick et al emphasise the importance of student engagement for monitoring 
quality by highlighting that it 
integrates what has been learned about quality of student effort, 
student involvement, and principles of good practice in undergraduate 
education into a broad framework for assessing quality and guiding its 
improvement. In this regard, it represents precisely what some leading 


scholars have argued has been lacking in higher education research 
(McCormick et al 2013:81). 


Promotes the development of desirable graduate attributes or higher education 
outcomes 

Various studies in the United States have linked student engagement to the development 
of graduate attributes on desirable higher education outcomes. The most prominent 
of these studies is the Wabash National Study of Liberal Arts Education (WNSLAE), 
which was a large-scale, longitudinal study (2006-2009) to investigate factors that 
affect liberal arts education outcomes. The WNSLAE and other research identified 


the following list of attributes promoted by student engagement: 
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= Persistence: As stated earlier, student engagement leads to better academic 
performance and therefore persistence in higher education (Kuh et al 2008). 

= Critical thinking: Student engagement promotes the development of critical thinking 
since it measures academic challenge, amount of reading and the emphasis 


institutions place on academic support and the promotion of interaction between 
students of different cultural backgrounds (Carini, Kuh & Klein 2006). 


= Moral reasoning: An analysis of the WNSLAE data shows that deep approaches to 
learning as measured by the NSSE has a modest, positive effect on the development 
of moral reasoning in first-year students (Mayhew, Seifert, Pascarella, Nelson Laird 
& Blaich 2012). 

= Need for cognition: The analyses in the WNSLAE show that interaction with staff as 
well as meaningful discussion with peers from diverse backgrounds helps to satisty 
student desire to engage in cognitive activities (McCormick et al 2013). 


Having reflected on the benefits of using student engagement surveys and data, 
| end this chapter by providing an overview of the book. 


OVERVIEW OF CHAPTERS 


The book is structured in three parts. Part | provides an overview of engagement 
work in South Africa. The second part focuses on creating institutional conditions 
impacting student success, and Part Ill focuses on how we can create conditions 
within the classroom that are conducive to student engagement and success. The 
book’s outline is as follows: 


Part I: Student engagement in South Africa: A status report 


Chapter 2 introduces the South African Survey of Student Engagement and provides 
an overview of what the student engagement landscape looks like in South Africa. 
The chapter also illustrates how student engagement data can be used to develop a 
systemic level understanding of the student experience. Intersectional analysis is used 
to map the learning experience of students by race, gender and generation status. 
The chapter raises important questions stemming from the data and concludes with 
suggestions of systemic level interventions that should be considered to enhance 
student learning and performance. 


Chapter 3 focuses on how student engagement data can help us understand and 
support students as they enter higher education institutions. Intersectional analyses 
from the Beginning University Survey of Student Engagement (BUSSE) data, as well 
as complimentary data from a recent financial stress scale added to the SASSE, help 
to highlight important challenges these students face. From the analyses, the chapter 
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makes certain recommendations on how institutions could intensify a data-driven 
focus to help first-year students succeed. 


Chapter 4 highlights the urgent need to develop the next generation of academic 
staff. The chapter considers how student engagement data can provide a data- 
driven insight into how more and less experienced staff approach teaching and 
learning and how their perspectives differ from the students. The chapter draws on 
data from the LSSE and gives recommendations on how academic staff development 
could be enhanced to build capacity in the system. 


Chapter 5 aims to unpack the conceptualisation and impact of high-impact practices 
(HIPs) implemented in the United States and discusses the potential of developing 
and measuring HIPs to contribute to the development and success of students in 
the South African context. The chapter shares data on the prevalence of HIPs as 
measured through the SASSE in an effort to contribute to the national debate on 


these practices. 


Part Il: Creating institutional conditions that matter for student success 


Chapter 6 outlines strategies for leading evidence-based improvement of student 
engagement. The first part of the chapter uses the role or designation as an 
international vehicle for engaging people in evidence-based change. It explores 
perspectives of quality assurance professionals, institutional researchers, department 
chairs, librarians, first-year experience coordinators, academic advisors, and careers 
advisors. The second part of the chapter takes a broader quality-improvement 
perspective and explores several ways in which institutions can use survey results to 
prompt change. 


Chapter 7 considers the role engagement data can play in assisting student 
affairs to advance student learning. The chapter begins with understanding the 
philosophical origins of student affairs in both the United States and South Africa 
and then discusses the evolution of student learning as a central tenet in the work 
of student affairs practitioners. The chapter gives examples of how student affairs 
educators could make use of student engagement data for improvement of student 
affairs. The authors also propose how a focus on student engagement can create 
new collaboration opportunities between curriculum learning, student affairs and 
academic development in the South African context. 


Chapter 8 addresses the key role of postsecondary leaders and managers at all 
levels in using information about student engagement to help promote and realise 
higher levels of student performance and institutional effectiveness. The authors 
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briefly describe how the conceptions of the role of educational leadership and 
management have changed over past decades. This backdrop is important because 
as it alters the nature of the responsibilities leaders have to their institution and 
students. This description is followed by a discussion of what leaders and managers 
can do to enact student engagement as an institution-wide strategy to induce 
students to take part in activities inside and outside the classroom that will help them 
attain the essential 21st century learning outcomes. 


Part IIl: Creating conditions for success inside the classroom 


Chapter 9 outlines the student engagement practices that matter for student learning, 
and focuses on those that instructors can influence. The chapter then explores the 
foundational background and concept of student engagement within teaching and 
learning spaces. This is followed by a summary of research findings highlighting 19 
student engagement and instructional practices that positively contribute to success. 
The chapter also explores how effective pedagogical practices could be contextualised 
in South Africa, and how or whether such practices impact academic achievement. 
Finally, the chapter considers the implications for pedagogical practices. 


Chapter 10 illustrates how student engagement data at the course/module level 
using the Classroom Survey of Student Engagement (CLASSE) can be used to 
create effective teaching and learning environments and promote evidence-based 
improvement in pedagogy. The chapter illustrates how the data can give students 
a voice (agency) in how they are taught, how the data can be used to empower 
academics to be better at teaching and how evidence can help to facilitate more 
collaborative relationships between students and their teachers. This chapter gives an 
in-depth exploration into why students and staff tend to generally ‘miss’ each other 
and highlights the use of the CLASSE measure for an evidence-based approach to 
academic staff development. 


Chapter 11 reflects on what we have learnt from student engagement data to date. 
The chapter provides recommendations on how institutions could further foster a 
culture of evidence; create, implement and evaluate interventions to support and 
develop students; contribute to a growing knowledge base on engagement and 
student success in general; as well as contribute to the development of engaging 
teaching and learning spaces. 
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PART | 


DEVELOPING A SYSTEMIC 
UNDERSTANDING OF THE 
STUDENT EXPERIENCE 


Francois Strydom & Cheryl Foxcroft 


There can be no peace without understanding — Senegalese proverb 


INTRODUCTION 


The Department of Higher Education and Training (DHET) emphasises the vital 
role that higher education plays in the development of a country and in changing 
peoples’ lives by helping them to break free from poverty. This should also be 
interpreted within the context of the National Development Plan: Vision for 2030, 
which was developed by the National Planning Commission (NPC) in collaboration 
and consultation with South Africans from all walks of life (NPC 2011). NDP 2030 
views higher education as a critical stakeholder in creating a virtuous cycle of growth 
and development to eliminate poverty and to sharply reduce inequality. 


These aspirations of the NPC stand in stark contrast with the realities of the inefficiencies 
plaguing the undergraduate education system. A DHET cohort analysis from 2000 to 
2008 of first-time entering undergraduate students in public higher education institutions 
(HEls) revealed that 20% of students entering in the 2005 cohort in contact mode had 
graduated after three years of study, 57% after six years of study, and 64% after ten years 
of study. In comparison, only 9% of students entering in the 2005 cohort in distance mode 
had graduated after six years of study, and 15% after ten years of study (DHET 2016). 


From an equity perspective, this cohort analysis found that black African and coloured 
students fared very poorly when compared to their Indian and white counterparts 
and that females outperformed males (DHET 2016). Moreover, the DHET (2017) 
reports the average graduation rate for undergraduate degrees at 17%, and the 
Council on Higher Education (CHE 2013) estimate that only half of students gaining 
access to higher education will ever graduate. 
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These sobering statistics, as well as the gap between aspirations for a better life higher 
education could provide for families, and the harsh reality of students struggling to 
succeed, urge us to find better ways to help students succeed. Our contribution to 
this urgency is to advocate for the potential of using evidence-informed analyses and 
interventions to develop the higher education system. Whereas Chapter 1 provides a 
conceptual introduction to student engagement and introduces the different measures 
that are used in the South African context, this chapter also demonstrates how student 
engagement data can be used to develop a comprehensive understanding of the 
student experience. The chapter gives an intersectional analysis to map the learning 
experience of different student groups by race, gender and generation status. It 
concludes by suggesting systemic level interventions that should be considered to 
enhance learning and performance. 


WHAT IS THE VALUE OF A SYSTEMIC PERSPECTIVE ON STUDENT 
ENGAGEMENT? 


The previously mentioned cohort analysis is an example of why increased 
accountability demands are being placed on higher education institutions. 
Internationally, public accountability demands increasingly expect HEls to find ways 
of providing evidence in concrete, observable, and measurable ways about what 
they are doing to ensure and enhance the quality of undergraduate education and 
student success (McCormick 2009). Higher education planning, policy and practice 
are also increasingly focused and reliant on how students think, behave and learn 


(Coates & McCormick 2014). 


The #FeesMustFall campaign and similar campaigns in the United States, United 
Kingdom and Europe have emphasised the importance of recognising and including 
student voices in decision-making and challenging how higher education is financed. 
Students, and providing them with a ‘data-driven’ voice, lie at the centre of student 
engagement research. 


Student engagement integrates theoretical and empirical higher education 
perspectives stretching back to the 1930s to inform contemporary good practices 
in undergraduate education and focuses on improving the quality of undergraduate 


education (Coates & McCormick 2014; Kinzie, McCormick & Gonyea 2016). From a 
systemic perspective, surveys of student engagement provide: 


= a more nuanced data-informed understanding of how different groups of students’ 
experience and engage in higher education; 


= data on educational behaviours and practices that are associated with desirable 
and more equitable outcomes for higher education systems, such as better 
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throughput and success rates, but also with quality graduates who are able to gain 
employment and contribute to economic development; 

= a perspective that is focused on student and staff behaviour instead of their 
satisfaction or attitudes and beliefs; 

= an opportunity for comparisons across different institutions as a result of the use of 
largely representative samples and standardised administration protocols; 

= information that bridges the gap between research and practice and allows system- 
wide and institutional leaders to develop action plans to enhance effective education 
behaviours by students and effective educational practices by institutions; and 

= a common language that can be used to further intra-and inter-institutional 
conversation and collaboration on how to address challenges relating to quality 


undergraduate education and to share possible solutions relating to these 
challenges (McCormick & McClenney 2012). 


Student engagement data have the potential to give us a snapshot of what is 
happening in universities with regard to students’ educational behaviours, enabling 
us to create a common language about what should be happening in our universities. 
This includes: to ensure quality education; to benchmark between institutions; and 
to generate targeted, systemic interventions based on the diagnostic results of the 
student engagement data. 


TOWARDS A DEEPER UNDERSTANDING OF STUDENTS 


To contribute towards the DHET’s idea of true transformation of South African higher 
education new ways of interpreting and using data need to be promoted. In this 
section, the results of the South African Survey of Student Engagement (SASSE) will 
be used to provide insights into the experience of different groups of students. The 
sample will first be described before presenting the results. The results have been 
disaggregated by generation status, gender, race and year of study to allow for a 
greater focus on intersectional analysis, which is a fairly new field in higher education 
research but has the potential to provide: 


= a more accurate reflection and understanding of the growing diversity of students 
in higher education; 


= a means of acknowledging voices and realities on the margins that may be silenced 
when we only focus on one-dimensional categories without consideration for the 
impact intersecting factors have on these students; 

= q means for uncovering how converging identities contribute to inequality (Griffin & 
Museus 2011) and this has implications for how we target institutional interventions; 
thus, intersectional analyses can enable higher education researchers to make 
prudent decisions about where to invest their energy; and 
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= richer understandings of how multiple social identities interact helps to avoid the 
challenge of advancing equality in one area while perpetuating inequalities in 
other areas (Wilson-Strydom, Strydom & Hen-Boisen 2016). 


When undertaking an intersectional analysis of survey data it is important to continue 
to ask questions about the meaning of the findings. This opens the possibility of 
further analyses being conducted to explore possible answers to the questions or 
even to conduct further studies. The sections that follow present the SASSE findings 
and pose possible questions that emerge from the findings. The sections also give 
reasons for posing these questions. It is hoped that this will provide some ideas of 
how to enrich our understanding of the SASSE findings and their meaning, which 
could assist in planning effective strategies to enhance student engagement. 


The results reported in this chapter are based on the SASSE 2014 survey, which 
included 12 306 respondents enrolled for a degree at nine institutions across 
South Africa. Four of these were traditional universities, two were comprehensive 
universities and three were universities of technology. The sample comprised 2 946 
first-year students and 9 360 senior students. A total of 47% of the respondents were 
males and 53% females. The average response rate per institution was 5.7%. 


The racial demographics of the respondents were 69% black African, 6% coloured, 
4% Indian/Asian, 16% white and 1% chose “other”. 


In the SASSE 2014 survey the faculties were grouped into four categories: 


= Business, Economics and Management 
= Human and Social Sciences 
a Sciences, Engineering and Technology 


= Education 


Approximately 34% of the students who completed the survey were enrolled in 
Business, Economics and Management, 16% were enrolled in Human and Social 
Sciences, while the majority (45%) of the sample were enrolled in Sciences, 
Engineering and Technology. Only 5% were enrolled in Education. 


THEMES AND INDICATORS 


The purpose of the themes and indicators, like the benchmarks reported for previous 
versions of the SASSE, is to provide a more manageable and digestible overview 
of the survey, which consists of 104 student engagement items (McCormick & 
McClenney 2012). The advantage of the themes and indicators in the updated 
version of SASSE is that they provide a more coherent framework and specific, 
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psychometrically robust measures for improving teaching and learning. These 
themes and indicators (Table 2.1) can be displayed in a dashboard that can be used 
to catalyse conversation between different stakeholders and institutions. 


TABLE 2.1 Themes and engagement indicators in SASSE 


e Higher-Order Learning 
Academie Challenge ° palace and integrative Learning 
e Learning Strategies 


e Quantitative Reasoning 


idea with Peers e Collaborative Learning 
i 
9 e Discussion with Diverse Others 


e Student-Staff Interaction 
e Effective Teaching Practices 


Experience with Staff 


; e Quality of Interactions 
Campus Environment l ; 
e Supportive Environment 


To produce an indicator score, the response set for each item is converted to a 60-point 
scale. For example, Never = 0; Sometimes = 20; Often = 40; Very often = 60. 
Thereafter, rescaled items are averaged. Thus, a score of zero means that a student 
responded at the bottom of the scale for every item in the engagement indicator, 
while a score of 60 indicates responses at the top of the scale on every item. First- 
year and senior students’ mean scores for the different indicators were compared 
by first-generational status, gender and race using Independent Samples t-tests. It 
is important to mention that although differences might seem subtle, it is important 
to keep in mind that we are working with aggregated averages and that student 
engagement comparisons are used to catalyse conversation and to highlight areas 
for further exploration. The mean scores of all indicators differ significantly between 
first-year and senior students (po = 0.000). A complete list of statistical and practical 
significance of all analyses in this chapter is included as Appendix A at the end of 
the chapter. Each of the themes and indicators will be examined next, followed by 
a discussion on the implications of the data for student development and success. 


Academic Challenge 


Challenging intellectual and creative work is central to student learning and academic 
quality. In particular, having high expectations is critical to motivate students and to 
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create an institutional culture that values academic achievement. To create more 
equitable outcomes it is vital that institutions create appropriate support to help 
students to reach these expectations (Kuh, Kinzie, Schuh & Whitt 2010). Universities 
promote student learning by challenging and supporting students to engage in 
various forms of deep learning that requires more than the mere memorisation of 
information. Four engagement indicators are part of this theme, namely: Higher- 
Order Learning, Reflective and Integrative Learning, Learning Strategies, and 


Quantitative Reasoning. 


Academic Challenge 
60 
50 
41 41 
40 a ail 
34 34 
5 
© 30 27 27- 
= 
20 
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Integrative Learning 
m First-year students m Senior students 


FIGURE 2.1 Engagement indicator mean scores for first-year and senior students regarding 
Academic Challenge 


Figure 2.1 shows that in terms of academic challenge, the experiences of first- 
year and senior students are very similar. It also shows that students engage more 
in Higher-Order Learning and Learning Strategies than Reflective and Integrative 


Learning, with the lowest mean scores indicated for Quantitative Reasoning. 


Higher-Order Learning 

The Higher-Order Learning indicator describes how much students’ academic work 
places emphasis on challenging cognitive tasks such as the application, analysis, 
judgment, and synthesis of information learned instead of just mere memorisation 
of facts (Kuh et al 2010). First-year and senior students’ mean scores for the Higher- 
Order Learning engagement indicator were compared by first-generational status, 


gender and race (Figure 2.2). 
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FIGURE 2.2 Higher-Order Learning mean scores for students by first-generational status, 
gender and race 


The disaggregation by race, gender and generational status shows only slight 
differences in the levels of higher-order learning. These differences are slightly 
more prominent for comparisons by race. The results indicate that black African 
students were slightly more engaged in higher-order learning than the other races. In 
addition, first-year white students and first-year “other” (representing a combination 
of racial classification options besides black African and white) students reported 
less emphasis on higher-order learning in challenging cognitive tasks than first-year 
black African students. Similarly, senior white students and senior “other” students 


reported less higher-order learning taking place than senior black African students. 


From an intersectional perspective, these findings highlight the need for a deeper 
interrogation of why the experience of first-year and senior students related to 
higher-order learning are so similar. A further question is whether analysing the data 
through the lens of other variables and using item level data instead of aggregated 
data might yield more meaningful information. For example, a post hoc item level 
analysis was undertaken using the level of challenge posed in subjects/modules, the 
extent of class preparation, and the extent of writing assignments required and how 
this impacted on the level of high-order learning emphasis experienced. This analysis 
showed that students who felt challenged by their subjects/modules experienced 
more emphasis on higher-order learning than students who experienced low levels of 
subject/module challenge. Furthermore, when students spent more time reading and 
preparing for class in modules/subjects where they felt challenged, they experienced 
more emphasis on higher-order learning. For senior students, higher levels of higher- 


order learning were associated with higher amounts of assigned writing they were 
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allocated. Overall, the more writing senior students were assigned, the more they 
perceived that their subjects/modules emphasised higher-order learning. 


Reflective and Integrative Learning 

Students are more likely to deepen and derive meaning from their studies if 
academic modules are structured so that the students can connect the content to 
their personal experiences and societal concerns. Lecturers who emphasise reflective 
and integrative learning motivate students to make connections between module 
content and various real-world examples. Helping students make these connections 
requires that they re-examine their beliefs and consider different perspectives and 
viewpoints other than their own. Consequently, this encourages reflective and 
integrative learning. Students who engage in reflective and integrative learning 
will also participate in deep approaches to learning (Laird, Shoup & Kuh 2005), 
which enrich their understanding of their academic subject/module content (Huber 


& Hutchings 2004). 


To determine students’ participation in reflective and integrative learning, first-year 
and senior students’ mean scores were compared by generational status, gender 
and race (Figure 2.3). 
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FIGURE 2.3 Reflective and Integrative Learning mean scores for first-year and senior 
students by first-generational status, gender and race 


Although Figure 2.3 displays significant differences between the reflective and 
integrative learning levels of first-year and senior students, in practice these differences 
are small. Senior students connect or combine module content to personal and 
prior experiences slightly more than first-year students. This shows that there is a 
difference between the experiences of first-year and senior students in the direction 
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we would expect — although one might ask whether the difference should be more 
substantial. Furthermore, the relatively low mean scores lead us to question whether 
the majority of students, and especially senior students, connect their learning to 
personal experiences and societal contexts. A post hoc analysis of the items in this 
scale shed some light on this question as it revealed that only 38% of first-year 
students and 46% of senior students reported that they regularly connected their 
learning to societal problems or issues. So, while a greater percentage of students 
could connect their learning to societal issues, it is worrying that not even half of 
the senior students experienced such connections. One possible reason for this 
was uncovered in the analysis in that only 37% of students indicated that diverse 
perspectives, where differing political, religious, racial/ethnic, gender and economic 
views could be offered, were raised or elicited in subject/module discussions or 
writing assignments. The post hoc analysis of the items also revealed that senior 
students experienced higher levels of reflective and integrative learning when they 
were required to write more and when they felt challenged to do their best work in 
the classroom. 


Learning Strategies 

University students improve their learning and retention when they actively engage 
with their subject material by analysing information as opposed to only memorising 
information. Effective learning strategies include summarising subject material, 
reviewing notes after class, and identifying key information in readings. Knowing 
how frequently students apply effective learning strategies can help universities target 
interventions to promote student learning and success and make a positive difference 
in many students’ degree attainments (e.g. Dumford, Cogswell & Miller 2016). 


Figure 2.4 shows that first-year students reported more frequent use of effective 
learning strategies than their senior counterparts. Similarly, senior non-first- 
generation students tend to use such learning strategies less frequently than senior 
first-generation students. Also, first-year females tend to make slightly more use 
of learning strategies than first-year males, but at senior level the same levels of 
learning strategies are reported for both the genders. Racially, first-year black African 
students reported more use of effective learning strategies than first-year white and 
first-year “other” students. Finally, senior “other” students reported the lowest use of 
learning strategies, while senior white students obtained the second lowest mean. 
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Learning Strategies 
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FIGURE 2.4 Learning Strategies mean scores for first-year and senior students by first- 
generational status, gender and race 


These findings raise interesting questions. For example, it is puzzling to understand 
why, if black African students report more frequent use of effective learning strategies 
than their white counterparts, differences in achievement levels between black 
African and white students persist? The DHET (2017) reports an 8% difference in 
student success between white and black African students, favouring the former 
(calculated as module passes divided by module enrolments and applying weighting 
according to race in this instance). Further research beyond just the use of learning 
strategies is clearly needed to better understand why the greater use of effective 
learning strategies by black African students compared to their white counterparts do 


not translate into more equitable outcomes in terms of performance. 


A post hoc item level analysis showed that most students (80%) reported that they 
regularly identified important information from reading assignments. Furthermore, 
most students described that they reviewed their notes after class, with 67% of first- 
year and 62% of senior students replied that they did so “Often” or “Very often”. These 
results point to a potential avenue of investigation, namely whether the academic 
success skills training in note-taking and academic reading is effective in changing 
the ways student learn. The analysis also identified that there is potentially a link 
between the use of learning strategies and academic performance. For example, 
students’ use of learning strategies was higher when their self-reported grades 
were also high. First-year students who reported their marks to be above 50% used 
effective learning strategies more often than those with lower self-reported marks. 
Senior students who reported marks of 80% or higher used learning strategies more 


frequently than those students with self-reported marks of 50-79%. 
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Quantitative Reasoning 

The ability to use and understand numerical and statistical information in day-to- 
day life is referred to as “quantitative literacy”. It has become increasingly important 
for university students to develop this ability through evaluating, supporting and 
critiquing the use of numerical and statistical information in real-life situations 
(McCormick, Gonyea & Kinzie 2013). With vast amounts of information available, 
much of which is in numerical form, through technological advancements, one 
cannot afford not to invest in the ability to make rational judgements about the 
credibility, use, and impact of this information. First-year and senior students’ mean 
scores for the Quantitative Reasoning engagement indicator were compared by first- 


generational status, gender and race, as presented in Figure 2.5. 
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FIGURE 2.5 Quantitative Reasoning mean scores for first-year and senior students by first- 
generational status, gender and race 


Figure 2.5 shows that comparisons by race point towards a slight difference between 
black African students’ reported quantitative reasoning skills when compared to 
the other races. White students and “other” students reported making less use 
of quantitative reasoning than black African students. Similarly, female students 
reported making less use of quantitative reasoning than male students. Senior male 
students reported the highest mean score for use of quantitative reasoning skills 
in this indicator when compared to the other groups. Interestingly, first-generation 
students reported a slightly higher mean for quantitative reasoning than non-first- 


generation students. 
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Learning with Peers 


Collaborating with others to master difficult material and develop interpersonal and 
social competence prepare students to deal with complex, unscripted problems they 
will encounter during and after university (Kuh 2007). Two engagement indicators 
comprise this theme: Collaborative Learning and Discussion with Diverse Others. 
Students’ reported learning with peers’ mean scores were somewhat different when 
compared by first-year and senior level (Figure 2.6). These differences will be teased 
out in the next two sections where each indicator is analysed in greater depth. 
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FIGURE 2.6 Learning with Peers engagement indicator mean scores for first-year and 
senior students 


Collaborative Learning 

Collaborative learning activities may include people working on group projects, 
asking peers for help with challenging subjects or explaining it to others, and going 
through subject material in preparation for exams within a group context. When 
asked how frequently students collaborated with other students, it was evident that 
not all students have equal engagement in collaborative learning. First-year and 
senior students’ mean scores for the indicator were compared by first-generational 
status, gender and race. 


Figure 2.7 shows that for all the comparisons performed, the mean for senior 
students was slightly higher than that for first-years. Furthermore, first-generation 
students reported a slightly higher mean than non-first-generation students. Of 
interest is that first-year male students reported less collaborative learning activities 
than first-year female students, but the mean scores for the senior males and senior 
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females were similar in terms of the collaborative learning opportunities engaged in. 
Additionally, white students and “other” students reported the lowest engagement 
in collaborative learning activities when compared to the higher mean score for the 
black African students. 
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FIGURE 2.7 Collaborative Learning mean scores for first-year and senior students by first- 
generational status, gender and race 


An important finding is that first-years are slightly less involved in collaborative 
learning than senior students. This finding suggests that first-year students might take 
a while to perceive the value of collaborative learning (i.e., the value only becomes 
clearer when they are in their senior years when they practice collaborative learning). 
First-years thus need to be intentionally reminded and involved in initiatives that 
allow them to learn collaboratively so that more of them develop an understanding 


of the value of such engagement to enhance their learning. 


Discussion with Diverse Others 

Universities have become places that offer students from various backgrounds 
and life experiences an opportunity to meet and engage with each other. These 
interactions are of mutual benefit to students and prepare them for personal and 
civic participation in diverse working and social environments (Crutcher, Corrigan, 
O'Brien & Schneider 2007). Student responses indicate that most students reported 
moderate levels of interactions. First-year students reported somewhat more frequent 
discussions with diverse others than their senior counterparts. 


Figure 2.8 shows that first-generation students are likely to interact less with diverse 
others than non-first-generation students. Also, first-year and senior males describe 
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a little more interaction with people from various backgrounds than females, while 
first-year and senior females have the same level of reported interaction with diverse 
people. This is the only engagement indicator where black African students obtained 
a lower mean score than the other racial groups. Black African students reported 
less contact with people from various backgrounds than white and “other” students. 
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FIGURE 2.8 Discussion with Diverse Others mean scores for first-year and senior students by 
first-generational status, gender and race 


A more fine-grained analysis of the items for this indicator revealed that 63% of 
students reported interacting regularly with people of a race or ethnicity other than 
their own. Furthermore, most first-year students (70%) and senior students (68%) 
stated that they “Often” or “Very often” had discussions with people from a different 
economic background than their own. Most students (65%) reported that they 
interacted regularly with people from different political and religious beliefs than 
their own. In terms of creating more equitable outcomes and the importance of 
facilitating diverse interaction, the analysis showed that both first-years and seniors 
who more frequently interacted with diverse peers also perceived their campus 
environment to be slightly more supportive and had more positive interactions 
with students, lecturers and staff. These more nuanced findings provide pointers to 
building a greater sense of social cohesion among students in that a more inclusive, 
holistic approach is required. 
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Experience with Staff 


Students learn first-hand how experts think about and solve problems by interacting 
with staff members inside and outside of instructional settings (Kuh et al 2010). As 
a result, staff become role models, mentors, and guides for lifelong learning. In 
addition, effective teaching requires that staff deliver course material and provide 
feedback in student-centred ways. Two engagement indicators investigate this 
theme: Student-Staff Interaction and Effective Teaching Practices. Students’ reported 
experience with staff mean scores varied somewhat when compared by first-year 
and senior level (Figure 2.9). These findings will be more fully unpacked in the two 
sub-sections dealing with each of the engagement indicators linked to this theme. 
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FIGURE 2.9 Reported mean scores for first-year and senior students concerning Experience 


with Staff 


Student-Staff Interaction 

Evidence suggests that students who have regular contact with lecturers and support 
staff are positively influenced by increasing students’ cognitive growth, engagement, 
development and academic success (Pascarella & Terenzini 2005). University staff 
have informal and formal roles as mentors, advisors and teachers, which models 
intellectual work, the effective use of knowledge and skills, and assist students to make 
conclusions regarding their studies and their future plans. As shown in Figure 2.10, 
the Student-Staff Interaction engagement indicator mean scores reported by first- 
year and senior students were low. 
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FIGURE 2.10 Student-Staff Interaction mean scores for first-year and senior students by first- 
generational status, gender and race 


The results were somewhat different when first-year and senior students were 
compared, as senior students tended to interact more with staff than first-year students. 
Furthermore, first-year and senior first-generation students reported higher student- 
staff interaction than first-year and senior non-first-generation students. Likewise, 
first-year and senior male students appear to be more comfortable interacting with 
staff than first-year and senior female students. Of everyone surveyed, black African 
students described more frequent contact with staff when compared to the other 
races. First-year white students obtained the lowest mean score for interacting with 
staff, while the first-year and senior “other” students did not differ in terms of their 


level of interaction with staff. 


An instructive question raised by these findings is what do students interact with 
staff about? An analysis of the items showed that only 21% of students “Often” or 
“Very often” discussed their career plans with a lecturer. Very few first-year students 
(18%) and senior students (21%) indicated that they work with a staff member on 
activities other than academic work, such as on committees, projects and student 
groups. Slightly more students reported discussing their past academic performance 
with a lecturer, with 26% of first-years and 24% of senior students doing so. Not 
surprisingly, the analysis showed that first-year and senior on-campus students were 
more likely to interact with lecturers than students living off campus. 


In general, students reported very low interaction levels with staff, with a mean of 
15.56 (out of a possible maximum score of 60) for first-year students and a mean 


of 17.82 for senior students. Considering the research on the important role that 
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academic and other staff play in the lives of student as role models, especially for 
first-generation students, these results beg the question of how interaction between 
staff and students can be improved. In addition, could facilitating greater interaction 
between staff and students help to attract more students into postgraduate studies 
and an academic career, thereby helping to address the capacity constraints in the 


system? Answers to these pertinent questions need to be grappled with. 


Effective Teaching Practices 

Effective teaching practices play an important role in facilitating student learning 
(Ambrose et al 2010). These practices promote student learning and skills and 
include the following activities: clear explanations, organised teaching, illustrative 
examples, and quick feedback regarding tests and assignments. First-year students 
reported that their lecturers made more frequent use of effective teaching practices 
than the senior students did (Figure 2.11). 


The figure shows that first-year and senior first-generation students’ mean scores 
for effective teaching practices were similar, but were higher than those for first- 
year and senior non-first-generation students. The mean score for first-year female 
students was higher than that of senior female students, while, first-year and senior 
male students had the same mean scores for this indicator. However, results indicate 
that black African students’ mean score was much higher than those of white and 


“other” students. 


Effective Teaching Practices 
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FIGURE 2.11 Effective Teaching Practices mean scores for first-year and senior students by 
first-generational status, gender and race 
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Subsequent analyses revealed that most students (78%) indicated that their lecturers 
clearly explained subject/module outcomes and requirements “Quite a bit” or 
‘Very much.’ Most students (81%) reported that their lecturers regularly presented 
subject/module sessions in an organised way. Fewer students (64%) indicated that 
lecturers provided feedback on a draft or work in progress. In addition, 66% of 
students stated that detailed feedback was provided in a timely manner. These 
findings provide insight into what aspects can be worked on to enhance the use of 
effective teaching strategies. 


Campus Environment 


Students benefit from and are more satisfied by supportive campus environments that 
cultivate positive relationships among students, lecturers, and staff (Kuh et al 2010). 
Two engagement indicators investigate this theme: Quality of Interactions and 
Supportive Environment (Figure 2.12). This will be explored further when the findings 
for the two indicators are presented in the sub-sections below. 
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FIGURE 2.12 First-year and senior students’ engagement indicator mean scores regarding 
their Campus Environment 


Quality of Interactions 

Positive interpersonal relationships support student learning and success, but the 
campus environment influences both the students’ relationships as well as their 
learning and success at university. Students who seek support from peers, advisors, 
lecturers and support staff are more equipped to find assistance when needed, 
and learn from those around them (Kuh et al 2010). The Quality of Interactions 
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engagement indicator was compared by first-generational status, gender and race 
(Figure 2.13). 
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FIGURE 2.13 Quality of Interactions mean scores for first-year and senior students by first- 
generational status, gender and race 


First-year students experienced a slightly higher quality of interactions at university 
than senior students reported. It is encouraging to see that first-year and senior 
first-generation students’ mean scores for Quality of Interactions at university were 
higher than the first-year and senior non-first-generation students, who obtained 
the same mean score. Furthermore, first-year and senior males, as well as first-year 
females had the same mean score for Quality of Interactions, while senior females 
obtained a slightly lower mean score for this indicator. When compared by race, 
first-year white students and first-year and senior “other” students had a lower mean 
score for Quality of Interactions at university when compared to the black African 
students. In contrast, first-year first-generation students and first-year black African 
students obtained the highest mean for the Quality of Interactions with students, 


peers and staff. 


With whom do students have quality interactions? The item level analysis showed 
that most students (88%) rated the quality of interactions with other students as 
“Good” or “Excellent”. Additionally, students’ interactions with academic staff and 
lecturers, as well as peer learning support providers such as tutors, mentors and 
facilitators were appreciated, with 71% of students reporting the quality of these 


interactions as “Good” or “Excellent”. 
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However, students reported a lower Quality of Interactions with staff in student support 
services and administrative services when compared to the other stakeholders. 
In addition, only 53% of first-year students and 51% of senior students rated the 
administrative services, such as registration and financial aid offices, as “Good” or 
“Excellent”. At a systemic and institutional level this finding provides an opportunity 
to reflect and possibly develop interventions to capacitate administrative and support 
service staff regarding how to work with students but also to find ways to help 
students understand the complexity and pressure that staff in these roles face within 
higher education. 


Supportive Environment 

Institutions that are dedicated to enhancing student success should aim to provide 
support to students across a variety of areas that include the cognitive, social and 
physical, and should encourage a high level of student performance and satisfaction 
(Pascarella & Terenzini 2005). This engagement indicator summarises students’ 
perceptions about the institution’s efforts regarding their learning and development. 
Figure 2.14 compares the Supportive Environment engagement indicator by first- 
generational status, gender and race. 
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FIGURE 2.14 Supportive Environment mean scores for first-year and senior students by first- 
generational status, gender and race 


First-year students reported a slightly more supportive environment than senior 
students. This suggests that a focus on the first-year experience has a positive impact. 
It also suggests that more focus is needed on senior students to help them deal with 
their academic and non-academic responsibilities and their ongoing development 


as learners. 
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In general, first-generation students perceived a higher level of support from their 
institution than that of non-first-generation students. Similarly, first-year females 
experienced more institutional support than first-year and senior males and senior 
females. Moreover, when compared by race, first-year and senior black African 
students experienced higher levels of support at their universities than white and 
“other” students did. Students living off campus found the environment less supportive 
than on-campus students did. 


What aspects of the institutional environment did students view as being supportive? 
Item level analysis indicated that students felt that their institution stressed the use 
of learning support services such as tutoring services, peer mentoring, writing 
centres and the library “Quite a bit” or “Very much”, with 81% of first-years and 
77% of senior students endorsing these categories. Most first-year students (64%) 
and senior students (59%) felt that their institution encouraged contact among 
students from different backgrounds (such as on social, racial/ethnic, religious, and 


economic levels). 


Further, it was evident that very few students (37%) believed that their institution 
emphasised helping them to manage their non-academic responsibilities, which 
included family and work matters. Less than half of first-year students (43%) and 
senior students (47%) stated that their institution encouraged them to attend events 
that addressed important economic, political, or societal issues “Quite a bit” or 
“Very much”. 


The findings reported in this sub-section provide much food for thought regarding 
how to address challenges to the campus environment and culture ranging from 
the need to provide a more supportive environment for senior students, to the need 
to train administrative and support service staff to effectively interact with students, 
to how to assist students to manage family and work-related matters. Greater 
innovative thinking is required to rise to these challenges so that the chances of 
attaining success can be enhanced for the majority of students. 


WHAT PICTURE IS FORMED ABOUT WHAT IS HAPPENING IN HIGHER 
EDUCATION? 


The data presented here allow us to gauge our progress in creating educational 
environments conducive to student success and development. From an equity 
perspective, black African students seem to engage more than their other racial 
counterparts in all indicators, except for Discussion with Diverse Others. Similarly, 
first-generation students seem to find the campus environment supportive, and 
engage more with staff and peer learning. First-year students also indicate more than 
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seniors that they experience a supportive campus environment, apply more learning 
strategies, and experience effective teaching practices, which could be attributed to 
a widespread movement to increase transitional support for first-year students. 


While these findings are encouraging, others point to concerns which need to be 
addressed. For example, only slightly more than half of the general student sample 
indicated that they engage in reflective and integrative learning, and 32% do not 
engage in higher-order learning. Both indicators are essential learning practices to 
master if we were to educate students according to the type of graduates or learning 
outcomes expected of higher education nationally and globally. Furthermore, less than 
half of students indicated that they engage in quantitative reasoning. This indicator 
extends beyond whether students are able to do math and incorporates drawing 
conclusions from analyses, linking numerical information to real-world issues and 
evaluations of others’ numerical conclusions — all of which are important transferable 
skills. A final noteworthy finding is the lack of interaction between students and staff, 
which is measured through noting the frequency in which students have discussions 
with their lecturers about career plans, academic performance, subject topics outside 
of class, and working with staff members on different projects. Since lecturers could 
be viewed as the primary academic (and sometimes non-academic) contact between 
students and the institution, we need to find ways to encourage these interactions. 


In sum, what we can learn from the findings reported and subsequent questions 
posed in this chapter, is that South African higher education needs to reflect on the 
issues raised in the following paragraphs. 


Raising the level of challenge and quality of support 


The level of academic challenge that students, especially senior students, experience 
has to increase. The results support the positive relationship between reading and 
class preparation and higher-order learning. For senior students in particular, 
including more writing opportunities facilitates the development of higher-order 
learning, as well as reflective and integrative learning. To help students meet the 
increased academic challenge demands it is important that institutions provide 
students with high-quality academic skills development and support to enable them 
to develop and apply effective learning strategies. However, SASSE findings suggest 
that although certain groups apply useful learning strategies, the effectiveness of 
these strategies could be questioned if the academic performance of these groups 
does not show substantial increase. Alternatively, it might be that the generic 
strategies provided by academic development facilitators need to be embedded in 
disciplines and be reinforced in class by lecturers. 
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Facilitating diverse collaboration to expand support 


The analyses suggest that institutions need to be more intentional about encouraging 
and creating opportunities for students to participate in collaborative learning 
opportunities. Collaborative learning helps students to become part of learning 
communities, which is one of the high-impact practices linked to improving student 
success. By creating collaborative learning opportunities, institutions can facilitate 
student conversations with diverse others. The findings show that discussion with 
diverse others increases students’ perceptions of their campus environment as being 
supportive. Therefore, an intentional focus on creating these opportunities will not 


only further social cohesion, but also enhance students’ chances of success. 


Encouraging effective teaching practices and interactions with staff 


The findings show that senior students report experiencing less effective teaching 
practices. Students perceive effective teaching as being directly related to developing 
higher-order learning. Conscious efforts to encourage reflective and integrative 
learning should also accompany consideration of effective teaching practices. 
The very low levels of student-staff interaction point to the need for innovative 
approaches in current resource constrained environments. The use of more effective 
teaching practices, such as the flipped classroom approach, blended and online 
learning, peer-facilitated learning (Supplemental Instruction and tutorials) and the 
use of technology to assist in more effective feedback and marking, could create 
opportunities for more frequent and high-quality interactions during class time and 
outside of class in the online environment. This could also create space for the 
development of undergraduate research and community engagement initiatives 
which would further promote student-staff interaction. One could also question 
whether getting academics to participate in discipline-linked co-curricular activities 


would help to facilitate interaction between staff and students? 


Optimising resources to create success-oriented campus environments 


There appears to be a consistent negative perception of the relationship between 
students and administrative staff. The quality of these relationships poses a strategic 
opportunity for institution-specific reflection on what the drivers are behind this 
persistent perception. Does having survey findings available offer an opportunity for 
conversations between administrative staff and students about how communication 
and interaction could be improved? Could these conversations help create a more 


supportive environment despite resource constraints? 
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Redesigning the senior experience for more equitable outcomes 


Throughout the analyses it became clear that senior students experience less academic 
challenge and support. As these students comprise the majority of students in higher 
education, finding innovative approaches to support them in a resource constrained 
environment has the potential to create a significant shift in throughput rates. 


Deepening data analysis to enrich understandings of student engagement 


While there is a strong argument for gathering data on student engagement in 
learning through inter alia using the SASSE, an equally strong argument has been 
made throughout this chapter that the resultant data need to be comprehensively 
‘mined’. While high-level analyses of trends using summative variables (e.g., the 
engagement indicators of the SASSE) are useful, such analyses can be enhanced 
through the following: 


= Adopting an intersectional approach to compare groups of students within and 
across institutions using a range of grouping variables (e.g., race, gender, field 
of study, generational status, and year of study) and applying univariate and 
multivariate analysis techniques. 

= Asking questions about the meaning of the findings and conducting post hoc (i.e., 
further) analyses to explore these questions. 

* Drilling down to item level data as this provides nuances and very useful information 
that guide the development of both a meaningful understanding of the findings and 
effective intervention strategies. 

= Triangulating other institutional data (e.g., academic performance) with survey 
data which can validate and enhance findings. 

= Conducting further studies to explore questions that arise from the findings that 
can’t be answered based on existing data (e.g., why do black African students use 
more learning strategies than students from other groups and yet we still find that 
there is not equity of outcomes in terms of academic success across race groups?). 


CONCLUDING THOUGHTS 


The chapter initially raised the question of the critical role higher education should 
play in helping to address historic inequalities and furthering economic development. 
Higher education policy and efficiency indicators such as dropout, success and 
throughput rates indicate that South African higher education needs to do better. It 
was argued that student engagement data provide an important link between higher 
education research and practice through providing institutions with actionable 
data and information. The most important driver behind student engagement is to 
improve the quality of undergraduate education. The value of providing institutions 
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with reports and data files is that the data can be used as a catalyst for conversations 
by allowing institutions to make relative comparisons between groups of students and 
between different institutions. The relative nature of these comparisons is because 
survey data are not perfect and therefore the results always need to be carefully 
and contextually interoreted. These comparisons enable institutions to identify how 
resources can be deployed based on evidence, ideally where student engagement 
data is considered along with other institutional data, instead of making strategic 


decisions based on anecdotal information. 


As has become evident in this chapter, the use of student engagement and other 
institutional data to develop targeted support for specific students could offer a 
solution that could help to transform the outcome of South African higher education. 
Yet the data also unearthed pertinent questions that need further examination, for 
example, is higher education failing in a key task to encourage a sense of social 
justice, diversity appreciation and social cohesion? What could these and other data 
sources tell us about addressing achievement gaps? And how could we build on 
creating a culture of evidence which would ultimately contribute to higher education 
policy and systemic prioritisation of interventions to promote student success and 


development? 
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APPENDIX A 


Statistical significance of indicators between first-year and senior students 


Indicator Mean tars t Significance ea 
Males 
Higher-Order Learning eae ae a ae -19.117 0.000* 0.13 
Reflective and Integrative Learning B a 5 P o -20.746 0.000* 0.14 
Learning Strategies a ee ee 0.000* 0.05 
Quantitative Reasoning Sool eae ae | leven 0.000* 0.08 
Collaborative Learning oi a 0.000 0.17 
Discussion with Diverse Others ean tee a o 8.327 0.000* 0.05 
Student-Staff Interaction Sie a oe 0.000" 0.14 
Effective Teaching Practices ee a ol eee 0.000" 0.03 
Quality of Interactions Slee) ace 0.000* 0.03 
Supportive Environment eae a ones 0.000*| 0.03 
Females 

Higher-Order Learning e yee 5 a aa 1.013 0.311 0.01 
Reflective and Integrative Learning mis ee mi -0.974 0.330 0.01 
Learning Strategies ei | eee oe | eee 0.000%, 0.11 
Quantitative Reasoning ee ee ee 0.000* 0.03 
Collaborative Learning P EE 0.000* 0.08 
Discussion with Diverse Others o Pa, A -1.148 0.251 0.01 
Student-Staff Interaction ia a a | a 0.000 0.19 


F s : FY: 42.91 FY: 12.948 " 
Effective Teaching Practices SNR. 4133| SNR. 13972) 19-260 0.000 0.12 


2 ` FY: S7 FY: 13.246 š 
Quality of Interactions SNR. 3605| SNR. 12923) 12796 0.000 0.08 


e ; FY: 35.36 FY: 14.242 2 
Supportive Environment SNR. 33.69 SNR. 15152, 18-690 0.000 0.11 
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Cohen’s 


Standard t Significance d 


Deviation 


Indicator Mean 


First-generation 


Higher-Order Learning a a pe -14.074 0.000* 0.08 


Reflective and Integrative Learning oF nue a nea -18.310 0.000* 0.10 


. FY: 41.09| FY: 12.561 . 
Learning Strategies SNR: 40.25 SNR: 12.980) 12177 9.099% 19:07 


FY: 27.33 FY: 14.776 


Quantitative Reasoning SNR. 2774| SNR. 15532) "5043 0.000* 0.03 
. F FY: 37.99 FY: 12.562 m 

Collaborative Learning SNR. 39.56. SNR. 12.599] "23073 0.000 0.12 
A F : x FY: 37.50 FY: 16.632 ¥ 

Discussion with Diverse Others SNR 37.76 SNR. 15563) "2872 0.004 0.02 


FY: 16.05 FY: 13.052 


Student-Staff Interaction SNR. 1831| SNR. 14362) "31248 0.000* 0.16 
; ; ; FY: 42.33 FY: 13.200 

Effective Teaching Practices SNR. 4222| SNR: 13.584 1.543 0.123 0.01 
r i FY: 37.86 FY: 12.730 % 

Quality of Interactions SNR. 3720) SNR. 13086) 2322 0.000 0.05 


x : FY: 34.17 FY: 14.435 
Supportive Environment SNR: 3429| SNR. 14830) 71-987 0.113 0.01 


Non-first-generation 


FY: 40.25 FY: 12.558 


Higher-Order Learning SNR: 40.27, SNR. 13235) ©7103 0.918 0.00 
Reflective and Integrative Learning ae ee s as nee 1.428 0.153 0.01 
Learning Strategies eu any se ah 16.880 0.000* 0.14 
Quantitative Reasoning a ae rae rae 1.958 0.050** 0.02 


Collaborative Learning re ee ra a -11.351 0.000* 0.10 


Se FY: 41.45) FY: 14.723 . 
Discussion with Diverse Others SNR. 40.00 SNR. 15.247 1.560 0.000 0.10 


. FY: 14.45 FY: 13.843 š 
Student-Staff Interaction SNR. 1647| SNR: 14503) 17075 0.000 0.14 


; A ; FY: 40.87 FY: 13.612 a 
Effective Teaching Practices SNR. 38.641 SNR. 14334) [9-158 0.000 0.16 


; ; FY: 34.94 FY: 13.310 
Quality of Interactions SNR. 34.80! SNR: 12777) 1-216 0.224 0.01 


3 FY: 33.20 FY: 15.188 z 
Supportive Environment SNR. 30.37 SNR. 15063! 22336 0.000 0.19 
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Cohen’s 


Standard t Significance d 


Deviation 


Indicator Mean 


Black African 


Higher-Order Learning Cee ll PY 12:5021] -3 gð 0.000* 0.02 


SNR: 41.55 | SNR: 13.042 
Reflective and Integrative Learning ae a B P -9.214 0.000* 0.05 
Learning Strategies Se onl ae Cua 0.000, 0.11 
Quantitative Reasoning gio | ed ees 0.022} 0.01 
Collaborative Learning a. en Ps eo -19.672 0.000* 0.11 
Discussion with Diverse Others a vee na a -3.514 0.000* 0.02 
Student-Staff Interaction ei | sone 0.000" 0.16 
Effective Teaching Practices mie ee s n ae 13.011 0.000* 0.07 
Quality of Interactions Pas T 5 Oe es 7.850 0.000* 0.04 
Supportive Environment Ps ei Ris bi 7.571 0.000* 0.04 


White 


Higher-Order Learning A A 5 a a -12.245 0.000* 0.14 


Reflective and Integrative Learning a ye 5 its reeds -9.438 0.000* 0.11 


f , FY: 37.51| FY: 12.565 
Learning Strategies SNR: 38.21 | SNR: 13.375 


Quantitative Reasoning Mr a es -1.308 0.191 0.01 


-4.593 0.000* 0.05 


Collaborative Learning Aala a a 0.000* 0.06 


n ; 3 F FY: 41.67 FY: 14.829 
Discussion with Diverse Others SNR: 41.87, SNR. 15.451) 71-096 0.273 0.01 


. FY: 12.15) FY: 11.741 r 
Student-Staff Interaction SNR. 14.99| SNR 13.502) 19227 0.000 0.21 


: ; : FY: 36.41 FY: 13.546 a 
Effective Teaching Practices SNR. 35.68 SNR. 12954) 4623 0.000 0.05 


7 . FY: 35.16 FY: 12.960 7 
Quality of Interactions SNR. 36631 SNR: 12333) 2157 0.000 0.12 


š ` FY: 27.95 FY: 14.015 ug 
Supportive Environment SNR. 2708| SNR. 13387) 5247 0.000 0.06 
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Cohen’s 


Standard t Significance d 


Deviation 


Indicator Mean 


Other 


Higher-Order Learning a oe Ae n -6.706 0.000* 0.08 


Reflective and Integrative Learning ` e E -3.796 0.000* 0.04 


; , FY: 38.95 FY: 12.906 i 
Learning Strategies SNR. 36.60 SNR. 13446. 15007 0.000 0.18 


Quantitative Reasoning a as ae poe 0.796 0.426 0.01 


Collaborative Learning aoo a a -10.789 0.000* 0.13 


FY: 45.36 FY: 15.098 


Discussion with Diverse Others SNR. 4395| SNR. 14.568 7.931 0.000* 0.09 
Student-Staff Interaction anao a o D 0.650. 0.01 
Effective Teaching Practices Si eel: eee 0.000" 0.16 
Quality of Interactions ma eae s Ba o 11.279 0.000* 0.14 
Supportive Environment oea a a 0.000*| 0.25 


*p < 0.01; **p < 0.05 
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UNDERSTANDING AND 
SUPPORTING STUDENTS 
ENTERING HIGHER EDUCATION 


Michael Henn, Lana Hen-Boisen & 
Hanlé Posthumus 


There comes a point where we need to stop just pulling people out of 
the river. We need to go upstream and find out why they're falling in — 
Desmond Tutu 


INTRODUCTION 


Students in their first year of study face potentially sizeable academic and non- 
academic transitional challenges. According to Department of Higher Education 
and Training (DHET), attrition rates for first-year students in South Africa have been 
fluctuating between 33% and 25% in recent years (DHET 2016). Combined with 
other specified data such as the student engagement surveys, these statistics act 
as diagnostic indicators to where interventions are most needed. From a systemic 
South African perspective, there has been a significant shift towards evidence-based 
decision-making in an effort to bolster success rates, inform policy and identity 


strategic goals for higher education. 


Building on this shift towards an evidence-based culture to help students succeed, 
this chapter argues for a more nuanced, data-driven approach to identify ways 
to support students and foster higher levels of engagement. This will be done 
through describing the Beginning University Survey of Student Engagement (BUSSE) 
and discussing the potential contributions this source of information can make to 
understand first-year students’ experiences and expectations when entering higher 
education. This chapter also focuses on how institutions could better support these 
students if we have a better understanding of who entering students are and how 


their expectations align with the realities of undergraduate studies. 
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THE IMPORTANCE OF STUDENT SUCCESS IN THE FIRST YEAR OF UNIVERSITY 


Identifying how students can be supported throughout their first year of study has 
been an important focus for higher education researchers in the United States for 
over 30 years and many valuable publications have resulted from this effort. While 
significant progress has been reported in these publications, retaining first-year 
students is still a global challenge. For example, a United States National Student 
Clearinghouse report (2014) shows that 69% of first-year students who enrolled in 
2012 returned to any institution in 2013. That means that 31% of these students 
dropped out. Similarly, first-year dropout rates for Australian higher education range 
between 42% and 7.5% (Burke 2016), and the United Kingdom has reported dropout 
rates of between 19% and 7% for first-years (Paton 2014). In South Africa, first-year 
dropout rates vary between 33% and 25% (DHET 2016) and the Council on Higher 
Education (CHE) estimates that 60% of students who drop out of higher education 
do so during their first year of study (CHE 2013). Although many factors might 
impact students’ decision to drop out, research suggests that failure to negotiate the 
transition between school and university makes a significant contribution to the high 
dropout rates in the first year (Parker, Summerfelt, Hogan & Majeski 2004). 


Intensifying transitional challenges is a student’s generational status, or whether they 
are the first in their families to attend higher education. Qualitatively exploring a group 
of first-generation students’ transitional experiences entering higher education in 
Australia, O’Shea (2009) stresses the lack of legitimate information sources reaching 
students to help create a realistic view of what to expect. Instead, misconceptions 
and reflecting on their transitions as a ‘learning curve’ were prominent for this group 
of students. In fact, authors such as Pike and Kuh have found that for the most part, 
first-generation students are at a disadvantage in comparison to non-first-generation 
students and that “a disproportionately low number of first-generation students 
succeed in college” (Pike & Kuh 2005:276). However, transitional difficulties are not 
only limited to first-generation students. Of the 128 students qualitatively interviewed 
about transitional challenges in a South African study conducted by Wilson-Strydom 
(2015), most students, irrespective of their race, gender, and school background 
experienced a dissonance between what they thought university would be like, and 
the reality they found. Further, Wilson-Strydom found that students faced difficulty 
with developing an understanding of how the university system and culture works, 
including application processes, course selection, and qualifying for financial 
support. In a study of the psychometric properties of the contextualised BUSSE among 
other focus areas, Mentz (2012) also found that students tend to overestimate their 
preparedness for university. 
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First-year students having unrealistic and overoptimistic expectations about their 
experiences in higher education stretch as far back as the 1960s, which Stern 
(1966) identitied as the “freshman myth”. Students tend to enter higher education 
with overly positive expectations of themselves and of the university, which are often 
not well matched to their actual experiences during the first year. More recent studies 
(e.g. Gonyea, Kuh, Kinzie, Cruce & Nelson Laird 2010) have found that students’ 
expectations for participation in a broad range of activities during the first year are 
not met — and that they systematically overestimate their levels of engagement (Davey 
2010; Gonyea, Kuh, Kinzie, Cruce & Nelson Laird 2006). In addition, students 
tend to overestimate the amount of time they would spend studying, only studying 
approximately half the amount of time they had anticipated (Babcock & Marks 2010; 
Gonyea et al 2010; Kuh, Gonyea & Williams 2005). Studies in South Africa also 
reveal a general mismatch between students’ high school experiences and university 
expectations, where students at lower levels of academic achievement in school 
tend to be over-confident in their ability to adjust to university and perform well 
academically (Bitzer & Troskie-De Bruin 2004; Nel, Troskie-De Bruin & Bitzer 2009). 


There is hope though — studies in the United States have found that explicit interventions 
targeting student development in the first year of study have had a significant impact 
on key success indicators such as persistence, academic performance, and nurturing 
a sense of lifelong learning (e.g. Garza & Bowden 2014; Padgett, Keup & Pascarella 
2013; Pike & Saupe 2002). These findings emphasise the importance of the nature 
and quality of the first-year educational experience as a contributor to students’ 
success and are particularly promising because they highlight that regardless of 
a students’ pre-university characteristics (Reason, Terenzini & Domingo 2006), 
institutions can intentionally design first-year programmes that meet the needs of 
diverse and vulnerable students. 


That said, for institutions to counter unrealistic expectations and other transitional 
challenges entering students face, we need to understand who these students are 
and the challenges they face. Only then would we be able to align support and 
developmental interventions with students’ needs. This is not an easy task though, 
with thousands of first-year students entering higher education each year. Thus, more 
large-scale surveys such as the BUSSE are needed in conjunction with institutional 
data to help us understand who our students are. 


THE BEGINNING UNIVERSITY SURVEY OF STUDENT ENGAGEMENT 


The BUSSE survey was designed to provide institutions with data about their first-year 
entering students early in the academic year. It provides actionable data that enable 
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institutions to identify and address the needs of students as a cohort as well as on 
an individual level. Through providing a better understanding of who students are 
and how they could be supported BUSSE provides the opportunity to intervene and 
support early on in the student journey (Kuh et al 2005). 


The BUSSE consists of 110 items, contextualised from the Beginning College 
Survey of Student Engagement (BCSSE) used in the United States. BUSSE items are 
grouped into nine subscales or indicators (Table 3.1), which provide institutions with 
the opportunity to engage with the data in a structured form. Seven of the BUSSE 
indicators measure student expectations for their first year of study and two indicators 
measure the extent to which they feel their high school prepared them for university. 


TABLE 3.1 Subscales comprising the BUSSE survey 


High School Engagement 


High school engagement with analysis 


Quantitative Reasoning (QR) Akay f 
and numerical information 


Use of effective learning strategies 


Learning Strategies (LS) in highschool 


First-year 


Expectation to interact and collaborate 


Collaborative Learning (CL) with peers 


Expectation to interact and engage 
with staff 


Expectation to engage in discussions 
with diverse others 


Student-Staff Interaction (SSI) 


Discussion with Diverse Others (DDO) 


Students’ certainty that they will persist 


Expected Academic Perseverance (PER) in the focs otacademicadversily 


Expected academic difficulty during 


Expected Academic Difficulty (DIF) ihe fra yearol universiy 


Students’ perception of their academic 


Perceived Academic Preparation (PREP) 
preparation 


Student-rated importance that the 
Importance of Campus Environment (CAMP) | institution provides a challenging and 
supportive environment 


To explore how the BUSSE could help us understand first-year students’ perceptions 
of their preparedness for higher education and their expectations for their first year 
of study, we examine aggregated data from the 2015 BUSSE administration of six 
universities (N=3055). 
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Figure 3.1 shows the plot of the nine BUSSE mean scale scores. For comparison 
purposes, the items forming part of each indicator are calculated to represent a 
value out of 60, and then averaged with the other items in the same scale. From a 
descriptive perspective, the scales provide some insight into the self-reported grouped 
competencies and expected experiences of students that enter the institution for the 
first time. 


From an overview of the scales we see that for the most part, students feel that they 
have developed sufficient learning strategies throughout their high school years. 
In contrast, a much lower mean of 30 indicates that students are not as confident 
in their quantitative reasoning abilities developed in high school. Looking ahead 
to the first year of study, students seem to feel very positive about their academic 
preparation (M=50), their academic perseverance (M=47), and about the impact 
the campus environment could have on their studies (M=46). However, medium to 
lower mean scores show that students do not expect to have many discussions with 
diverse others, collaborative learning experiences, or interactions with staff members. 
Probably the most significant finding is the notable difference between how prepared 
students feel for university versus how difficult they think their first year at university 
will be (with mean scores of 50 and 25 out of 60, respectively). Considering that 
these scales are likely to reflect what students are familiar with and how they have 
previously experienced teaching and learning at school, the need to appropriately 
acculturate students and support them in managing academic and non-academic 
transitional challenges is clear. 


2015 BUSSE Scales 
60 
50 
50 a 46 
4] 38 
40 
s 30 22 30 
© 30 25 
= 
20 
10 
0 
Quantitative Learning Collaborative Student-Staff Expected Expected Expected Perceived Importance 
Reasoning Strategies Learning Interaction Discussions Academic Academic Academic od Campus 
nt Eee Perseverance Difficulty Preparation Environment 
ers 


FIGURE 3.1 2015 BUSSE scale scores 
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Figure 3.2 shows the results of an intersectional analysis to probe a bit deeper into 
which students, based on generational status, race and gender, indicate higher or 
lower mean scores out of 60 on the subscales. 


Regarding the two scales representing high school preparedness, all groups, 
irrespective of generational status, gender or race, indicate moderate to low 
perceptions of quantitative preparedness. Interestingly, white first-generation (FG) 
and non-first-generation (non-FG) males indicated significantly lower scores in 
the frequency of applying certain learning strategies. Both these groups of white 
males also indicated the lowest means in their expectations of collaborative learning 
(M=26 respectively), interactions with staff (FG M=24; non-FG M=23), and having 
a supportive campus environment (M=40 respectively). In general, white students, 
particularly females, expect to have more discussions with diverse others during their 
first year of study, while black African students in general expect to persevere more. 
Black African students in general see themselves as more prepared for the upcoming 
academic year, and do not expect it to be as difficult as their white counterparts do. 
The gap between expected difficulty of the year ahead and perceived preparedness 
is at its widest for black African, first-generation students (both male and female). 


TRANSITIONING FROM HIGH SCHOOL TO UNIVERSITY 


Students face certain difficulties regarding their transition to from high school to 
university. The BUSSE data are especially helpful in determining whether students 
feel prepared for the transition process, and how they expect their behaviour 
regarding academic and non-academic activities will change. This section considers 
some selected factors impacting on transition challenges and how students expect 
to engage with them, namely time management, student-staff interaction, perceived 
preparedness, and expected difficulties. 


Time management is one of the main priorities that a first-year student faces 
(Figure 3.3). Students are asked to report how much time they spent during their last 
year of high school preparing for class (studying, reading, writing, doing homework, 
rehearsing, and other academic activities), participating in co-curricular activities 
(organisations, school publications, student government, sports, etc.), and relaxing 
and socialising (time with friends, video games, watching TV or movies, sport, mobile 
and online chatting, etc.). Students are then also asked to estimate how much time 
they expect to spend on these activities during their first year of study. 


When comparing first-generation students, black African men and women expect to 
spend three hours more than they had spent in high school preparing for class, while 
white male and female students expect to spend six and five hours more respectively. 
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Black African male non-first-generation students expect to spend three hours more on 
preparing for class, while white non-first-generation men expect to spend seven hours 
more. Black African non-first-generation women expect to spend four hours more, 
while white non-first-generation women expect to spend three hours more. However, all 
female groups spent more time preparing for class during high school and expect to 
spend more time than the male groups preparing for class during their first year of study. 


Male and female white, non-first-generation students show the most difference 
in time they expect to spend on co-curricular activities, with males expecting to 
spend five hours per week less on these activities, and females four hours less. Black 
African non-first-generation male and female groups expect to spend the same 
amount of time on co-curricular activities that they spent during high school. While 
all students expect to spend less time on social activities, black African and white 
non-first-generation males show the biggest difference in time they expect to spend 
socialising, with three and four hours less respectively. In general, white students, 
irrespective of generational status or gender, expect to spend two to six hours more 
than black African students on social activities. 


The student-staff interaction items included for this chapter are: how often do you expect 
to talk about your career plans with a lecturer, work with a staff member on activities other 


than academic work (committees, projects, student groups, etc.), discuss your academic 
performance with a lecturer, and discuss subject topics, ideas, or concepts with a lecturer 
outside of class (Figure 3.4). Results are shown as the percentage of responses indicating 
expectations of engaging in such behaviours “Often” and “Very often”. 


In general, students do not expect much interaction with staff members during their 
first year of study and black African students, as well as first-generation students 
seem to expect to interact slightly more with staff. White males, particularly non-first- 
generation white males, expect least interaction with staff in terms of discussions about 
career plans, working with staff on projects, and discussing academic performance 
with a lecturer. White female students of both first and non-first-generational status 
show the least expectation to interact with lectures outside of class. 


Regarding perceived preparation, students were asked how prepared they are 
to write clearly and effectively, speak clearly and effectively, think critically and 
analytically, analyse numerical and statistical information, work effectively with 
others, use computing and information technology, and learn effectively on their own. 
Figure 3.5 shows that all students, irrespective of gender, race, or generational status 
believe that they are very well prepared to master their first year of study. In fact, all 
(100%) white female non-first-generation women indicated that they are completely 
prepared to learn effectively on their own. Slightly less confident responses are seen 


63 


SOujNIIYIP papedx] 9° FUNDS 


S4-UOU əjowa AYM E 94 Pwy yM xe D4-UOU BJOW AYM X D4 BOW ALY @ D4-UOU əjowa UDI Pog @H 4 jowa; udiyy 4209Y O4-uou ejow uDduyy yD0)g 94 ajow unsyyy 201g 


JOM DIWEpPDdD sasuadxe jOlUepou 
spuely meu Buoy YM day Buie Aysuaaiun Bulkog aul Nok Buou palqns Buus) 
0 
x > > > pod > 
E oz 
zz Z 
Lt ilor ca Fz S o£ 
ze E£ le lg > 
sensemo SE SE 9€ ee OV a 
tp OF OF lp m 6E 8 
oe 8r = OS s 
6r a 
ES. o 
as 09 
s 
a a9 OL 
06 
OOL 


CHAPTER 3 ° HENN, HEN-BOISEN & POSTHUMUS 


64 


UNDERSTANDING AND SUPPORTING STUDENTS ENTERING HIGHER EDUCATION 


regarding preparedness to analyse numerical and statistical information, with 76% 


of white male first-generation students being the least prepared group. 


Students were asked to indicate on a six-point scale how difficult they think certain 
scenarios will be during their first year of university, ranging from “Not difficult at 
all”, to “Very difficult”. These items include learning subject material, managing your 
time, paying university expenses, getting help with academic work, and making new 
friends. Figure 3.6 only presents responses between 3 and 6, representing the higher 
end of difficulty on the scale. 


Less than 40% of respective first-generation black African male and female students 
feel that learning subject material is going to be difficult during their first year of 
study. These two groups also stand out as not expecting that managing their time 
would be very difficult, with 41% of black African first-generation males and 36% of 
black African first-generation females indicating as such. While none of the student 
groups attribute particular difficulty to getting help with academic work and making 
new friends, white female non-first-generation students seem to attribute the least 


difficulty to these items. 


INFLUENCE OF FINANCIAL STRESS ON FIRST-YEAR SUCCESS 


An important national focus of higher education in South Africa since 2015 has 
been student frustration about university fees being unaffordable, with widespread 
#FeesMustFall protests shutting down many universities for periods of time. To 
give due attention to students’ challenges with fees, which impact significantly on 
retention, dropout and success, we add some first-year findings from the 2016 
administration of the South African Survey of Student Engagement (SASSE), which 


included a subscale on financial stress. 


The BUSSE data in Figure 3.6 show us that black African males, irrespective of their 
generational status expect most difficulty to pay university expenses, followed by black 
African female first-generation students, and white male first-generation students. 


The Financial Stress Scale (FSS) asks students to what extent they worry about paying 
for university. Figure 3.7 shows that financial concerns over paying university fees are 
common to most students, except for almost half of white male and female non-first- 
generation groups, who report that they never worry about paying fees. The groups 
most affected by worrying about how they are going to pay for studies are black 
African male and female first-generation students, of whom half indicate that they 
worry every day. In general, black African students worry more than white students, 
and first-generation students worry more than non-first-generation students. 
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FIGURE 3.7 Intersectional analysis of first-year students worrying about paying for university 


Students were asked about food security by indicating whether they have run out 
of money and are unable to buy more food. Analysing what the first-year students 
reported, Figure 3.8 shows that most of the students who indicated “Sometimes”, 
“Most days”, and “Every day” fall into the black African male first-generation and 
black African female first-generation groups. Notably, 36% of black African male 
first-generation students and 34% of black African female first-generation students 
indicate that they run out of food and cannot afford to buy more on most days 
or every day. Also noteworthy is the difference between black African students in 
general, as well as between first and non-first-generation students, where black 
African and first-generation students report higher incidences of running out of food 


without being able to buy more. 


The FSS also asked students whether they have considered dropping out because of 
financial reasons. Figure 3.9 shows an analysis of whether students who indicated 
that they worry about whether they will be able to pay day to day expenses; whether 
they will be able to pay for university; and whether they have run out of food without 
being able to buy more on most days or every day, have considered dropping out 
because of financial reasons. Too few white students were in the study to be included 
in this analysis. While black African female non-first-generation students show the 
highest frequency of consideration to drop out (63%), first-generation students show 
the lowest consideration of drop out. This means that even in the face of hunger and 


constant stress about finances, students are determined to persist in higher education. 
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FIGURE 3.8 First-year students who ran out of money and could not afford to buy more food 


Will students who worry about day to day expenses, paying for 
university, and not being able to buy food, consider dropping out? 
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FIGURE 3.9 Students who worry about fees, paying day-to-day expenses and who run out of 
food and consider dropping out 


DISCUSSION 


The data presented here help us to better understand the students entering higher 
education. The intersectional findings show us that white males are (or expect to 
be) less engaged, have fewer expectations to work with others, to interact with staff, 
and expect less of the campus environment to contribute to their success. Mentz 
(2012) reported similar findings, where white males engaged at significantly lower 
levels than all other groups at high school and expected to engage at significantly 
lower levels than all other groups during their first year. Mentz’s findings, as with this 
chapter’s findings, seem to confirm the “freshman myth”, referring to the unrealistic 
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underestimation of expected difficulties students perceive, combined with an 
overestimation of their preparedness for university. If then, as authors such as Garza 
and Bowden (2014), Padgett et al (2013), and Pike and Saupe (2002) have found 
that experiences in the first year can counter some pre-university characteristics and 
impact development towards success, we need to find out which experiences work 
in our context to try and level the very unequal playing field students find themselves 
on when entering university. 


This does not mean that we do not have to take the expectations of students lightly. 
The data presented here regarding black African male and female first-generation 
students are particularly troublesome. These students show the widest gap between 
perceived preparedness and expected difficulty; expect time management and 
learning subject material specifically to be less difficult; expect to spend more time 
on academics, but still less than other groups; and expect more interactions with 
staff. More importantly, these groups show the most vulnerability when it comes 
to financial stress — about paying for university fees — but also about where their 
next meal is going to come from. The data tell us that black African students, and 
particularly first-generation students who worry about paying day to day expenses, 
who worry about paying university fees, and who experience food insecurity are more 
determined to stay in university. The promise of what a better life higher education 
brings seems to be a very strong motivating factor for these students. 


HOW WE CAN USE DATA TO BETTER SUPPORT OUR STUDENTS TOWARDS SUCCESS 


Using data/evidence to inform practice is at the core of the student engagement 
surveys. Thus, for institutions to optimally align the use of BUSSE and other data 
sources with support structures, the first step would be to identify what we are aiming 
for. In other words, what do we mean by first-year student success. Upcraft, Gardner 
and Barefoot (2005:8-9) state that most institutions would define first-year student 
success as including one or more of the following: 

= Developing intellectual and academic competence (including critical thinking, 

problem solving, and reflective judgement). 


= Establishing and maintaining interpersonal relationships (particularly to help them 
with transitional challenges). 


= Exploring identity development. 

= Deciding on a career. 

= Maintaining health and wellness. 

* Considering faith and the spiritual dimensions of life. 
= Developing a multicultural awareness. 


= Developing civic responsibility (a sense of citizenship in a democratic society). 
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While this comprehensive definition is ideal, to do these developmental milestones 
justice means that measuring and tracking success would rely primarily on qualitative 
interpretations of the extent to which these milestones have developed throughout 
the first year of study. However, considering the complexities first-year students face 
with transitions, the developmental phase most of them are in, and how professional 
identities are shaped through interactions with knowledge, we need to find ways 
to move beyond measuring persistence to the second year of study and academic 


achievement as the sole quantitiable indicators for success. 


If our definition of success for students in their first year of study is broadened to 
include development in personal, social, cultural, and academic spheres, we can 


consider the following interventions to support them: 


= The disconnect between students’ expectations about their abilities and the 
ditticulties they can expect in their first year of study prompt us to reconsider how 
the institutional culture and campus environment could be used to facilitate the 
transition between expectations and realities. 


= Factors which might be inhibiting engagement during the first year need to be 
identitied in order to establish a more facilitating teaching and learning culture 
(Howard 2005). Research in this area can in turn inform staff development initiatives 
aimed at the renewal of teaching and learning practices that meet the needs of 
modern day student populations within the classroom context (Mentz 2012). 


= Data obtained from the BUSSE early in the students’ career can form part of a 
comprehensive early warning system to identity students at-risk and to design 
customised interventions. Through disaggregating data, institutional researchers 
can identity groups of students within faculties who report high or low expectations 
to engage at university and link these with other institutional data. The data 
could then be used to reach out to students and provide necessary referrals to 
support structures. 


= By identifying these trends and communicating the results through to teaching and 
learning structures the BUSSE results can be used to inform the type of support 
provided to targeted groups of students. For example, informing high-impact 
practices, such as first-year seminars, or forming learning communities. 


= BUSSE results could inform individualised support, such as generating academic 
advising profiles to help guide students toward reaching their academic and 
career goals. 

= Analysis of BUSSE data could also be used to inform academic and non-academic 
staff development. If staff are aware of the expectations and difficulties students 
enter higher education with, they might be more understanding when planning 
lecturers or explaining financial aid options. 
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= BUSSE data could also play a central role in planning first-year orientation 
programmes, student pathway programmes and academic support programmes. 

= When examining student retention, the focus should not only be on identifying the 
reasons why students drop out, but also on the reasons why some students chose 
to persist in their studies and how this persistence and engagement is enacted at 
an individual level (O’Shea 2009). 

= We still know very little about how student finances (or lack thereof) impact retention, 
dropout and success. A recent national report on the relationship between financial 
support on access and success in England, found that “students receiving financial 
support have comparable non-continuation rates with students who do not receive 
financial support. Yet ... students in receipt of financial support report that it has 
enabled them to stay on course and that they consider withdrawing less than 
their peers” (Nursaw Associates 2015:4). Challenges related to student poverty 
are often not solely solved through money. Much more research is needed to try 
and understand what students are going through financially for institutions, private 
organisations, and government to assist them. 


Each of these trends identified in the BUSSE data offer some information to institutions 
to answer the question “Who are our students?”, which can be meaningfully used 
to design teaching and learning environments that are conducive to success. 
Aggregated BUSSE data provide us with a deeper understanding of what certain 
groups of students expect when they enter higher education as well as how prepared 
they feel to manage these expectations. BUSSE also has the potential to contribute 
to personalised learning and support. From an equity perspective, looking at each 
individual student within their unique context can provide valuable insight into their 
lives, desires and expectations. Through focusing on the student as a unique person, 
the institution can better understand the challenges that students regularly face and 
customise directed intentional support structures to help these students overcome 
their challenges. To achieve this there must be institutional commitment towards 
data-driven decision-making. 


CONCLUSION 


Student engagement data provide the institution with a broader context of the 
students it serves. For first-year students, the BUSSE data enable an overview of which 
groups need more support, as well as a more nuanced approach to individualised 
decision-making. Integrating student engagement data with institutional data can 
enable the institution to act instead of reacting, further building on the notion 
that the ever-changing student perspective can be appreciated by focusing on 
the individual student’s needs in an effort to optimise the learning experience of 
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each student. The BUSSE scale analysis elucidated the differentiation in response 
characteristics amongst particular students and by exploring these characteristics 
it enables the institution to better understand its first-year entering students. The 
implications of student engagement data add to the knowledge of the field and is 
even more powertul when combined with other information about students collected 


at application and registration. 


REFERENCES 


Babcock, P & Marks, M. (2010). The falling time cost of college: Evidence from half a century 
of time use data. NBER Working paper 15954, National Bureau of Economic Research. 
Retrieved from http://www.nber.org/papers/w15954. https://doi.org/10.3386/w15954 


Bitzer, E. & Troskie-De Bruin, C. (2004). The effect of factors related to prior schooling 
on student persistence in higher education. South African Journal of Education, 
24(2):119-125. 


Burke, L. 2016. University attrition rates: Why are so many students dropping out? Retrieved 
from http://www.news.com.au/finance/work/careers/university-attrition-rates-why-are-so- 
many-students-dropping-out/news-story/3e49 1dd1 19e1249a5a3763ef80 1 Of8b5 


CHE (Council on Higher Education). (2013). A proposal for undergraduate curriculum reform 
in South Africa: The case for a flexible curriculum structure. Pretoria: CHE. 


Davey, C. M. (2010). Predicting college student success: A historical and predictive 
examination of high school activities and accomplishments. PhD dissertation, University of 
Utah, Salt Lake City, UT. 


DHET (Department of Higher Education and Training). (2016). 2000 to 2008 first time entering 
undergraduate cohort studies for public higher education institutions. Pretoria: DHET. 


Garza, E. & Bowden, R. (2014). The impact of a first year development course on student 
success in a community college: An empirical investigation. American Journal of 


Educational Research, 2(6):402-419. https://doi.org/10.12691/education-2-6-13 


Gonyea, R. M., Kuh, G. D., Kinzie, J., Cruce, T. & Nelson Laird, T. (2006). The influence of 
high school engagement and pre-college expectations on first-year student engagement 
and self-reported learning outcomes at liberal arts institutions. Paper presented at the 
annual meeting of the Association for the Study of Higher Education, Anaheim, CA (17 
to 19 November). Retrieved from http://cpr.iub.edu/uploads/Expectations%20and%20 
Engagement%20ASHE%202006%20Gonyea%20Kuh%20Kinzie. pdf 


Gonyea, R.M., Kuh, G.D., Kinzie, J., Cruce, T. & Nelson-Laird, T. (2010). Expectations and 
engagement: How liberal arts college students compare with counterparts elsewhere. 
Special report prepared for the Center of Inquiry in the Liberal Arts at Wabash College, 
Crawfordsville, IN. Retrieved from http://www.nsse.iub.edu/_2010/img/NewNI/Wabash__ 
Expectations Engagement. pdf 


Howard, J. (2005). Why should we care about student expectations? In: T.E. Miller, B.E. 
Bender, J.H. Schuh & Associates (Eds), Promoting reasonable expectations: Aligning student 
views of the college experience, 10-34. San Francisco: Jossey-Bass. 


71 


CHAPTER 3 ° HENN, HEN-BOISEN & POSTHUMUS 


Kuh, G.D., Gonyea, R.M. & Williams, J.M. (2005). What students expect from college and 
what they get. In: T.E. Miller, B.E. Bender, J.H. Schuh & Associates (Eds), Promoting 
reasonable expectations: Aligning student views of the college experience, 34-64. San 
Francisco: Jossey-Bass. 


Mentz, M. (2012). Measuring and using pre-university levels of student engagement at a South 
African university. PhD thesis, University of the Free State, Bloemfontein. 


National Student Clearinghouse. (2014). Firstyear persistence rate of college students declines. 
Retrieved from http://nscnews.org/first-year-persistence-rate-of-college-students-declines/ 


Nel, C., Troskie-De Bruin, C. & Bitzer, E. (2009). Students’ transition from school to university: 
Possibilities for a pre-university intervention. South African Journal of Higher Education, 
23(5):974-991. https://doi.org/10.4314/sajhe.v23i5.4881 1 


Nursaw Associates. (2015). What do we know about the impact of financial support on access 
and student success? Report to the Office for Fair Access (OFFA), UK. https://www.offa. 
org.uk/wp-content/uploads/2015/03/Literature-review-PDF. pdf 


O'Shea, S. (2009). Hearing the student voice — examining the processes of transition, 
persistence and engagement for a group of first year, first in family university students. In: 
S. Kennedy & C. Doyle (Eds), 12th Biennial CSSI confederation: The first year experience, 
82-93. Dublin: The Confederation of Student Services in Ireland. 


Padgett, R.D., Keup, J.R., and Pascarella, E.T. (2013). The impact of first-year seminars on 
college students’ life-long learning orientations. Journal of Student Affairs Research and 
Practice, 50(2):133-151. https://doi.org/10.1515/jsarp-2013-0011 


Parker, J.D.A., Summerfeldt, L.J., Hogan, M.J. & Majeski, S.A. (2004). Emotional intelligence 
and academic success: Examining the transition from high school to university. 
Personality and Individual Differences, 36:163-172. https://doi.org/10.1016/S0191- 
8869(03)00076-X 


Paton, G. (2014). More than 26,000 students ‘dropping out of university’. The 
Telegraph, 28 March 2014. Retrieved from http://www.telegraph.co.uk/education/ 
universityeducation/10727953/More-than-26000-students-dropping-out-of-university.html 


Pike, G.R. & Kuh, G.D. (2005). First- and second-generation college students: A comparison 
of their engagement and intellectual development. The Journal of Higher Education, 
76:276-300. https://doi.org/10.1080/00221546.2005.11772283 


Pike, G.R. & Saupe, J. (2002). Does high school matter: An analysis of Three methods of 
predicting first year grades. Research in Higher Education, 43(2):187-207. https://doi. 
org/10.1023/A:1014419724092 


Reason, R.D., Terenzini, PT. & Domingo, R.J. (2006). First things first: Developing academic 
competence in the first year of college. Research in Higher Education, 47(2):149-175. 
https://doi.org/10.1007/s11162-005-8884-4 


Stern, G.G. (1966). Myth and reality in the American college. AAUP Bulletin, 52(4):408-414. 
hitps://doi.org/10.2307/40223463 


Upcraft, M.L., Gardner, J.N. & Barefoot, B.O. (2005). Introduction. In: M. L. Upcraft, J.N. 
Gardner, B.O. Barefoot and associates, Challenging and supporting the first-year student: 
A handbook for improving the first year of college. San Francisco: Jossey-Bass. 


Wilson-Strydom, M. (2015) University access and success: Capabilities, diversity and social 
justice. London: Routledge. 


72 


BUILDING ACADEMIC 
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Learning expands great souls — Namibian proverb 


INTRODUCTION 


The need to develop the next generation of academics and research capacity is 
particularly urgent on the African continent and in South African higher education 
(Denney, Mitchell & Wheat 2015; Department of Higher Education and Training 
[DHET] 2017). This capacity has to be developed in a higher education context 
characterised by pressure of transformative emergent technologies, scarce and 
constrained (human and financial) resources, lack of student financial aid and 
increased accountability demands based on concerns about the efficiency of higher 


education from government, private companies and donors (Kuh et al 2015). 


Quinn (2012) points out that higher education has the responsibility to substantially 
improve its effectiveness and efficiency. What is required is to reduce wastage and 
improve the educational process within the system. This would depend on recognising 
the importance of effective teaching and education development support as the core 
business of an institution and the reconceptualisation of recruitment and promotion 
systems to differentiate and recognise the contribution that staff with a focus on 
research and those that focus on teaching and learning can bring (Scott 2009). 
In addition to improving the quality and status of teaching, South Africa, like other 
countries, is confronted with an ageing academic workforce. There have also been 
many calls for the transformation of the composition, size and capacity of academic 
staff, which has not changed significantly enough to represent post-apartheid 
South Africa. Further, given the complexity of the role of academics today, where 


good teaching, research and social responsiveness are expected, coupled with the 
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recruitment of sizeable cohorts of new young staff and the demonstrable current 
inefficiencies of the educational processes, systematic improvements in respect of 
the effectiveness of lecturers in their roles as educators must be accorded a higher 
priority than has traditionally been the case. 


Such recognition of the role of educators will also highlight other pivotal roles that 
academic staff play in higher education systems, such as role models, mentors 
and guides (Kuh, Kinzie, Schuh & Whitt 2010). That said, there is limited research 
focusing on lecturers’ expectations of students’ learning, their general teaching and 
learning practices, and on the need to align the lecturer-student relationship to 
promote student success, especially in South Africa. Considering the vital importance 
of such matters in the higher education project and the need to identify avenues 
where development initiatives could be targeted, this chapter will focus on helping 
to develop an evidence-based understanding of how lecturers today think about 
and practice effective teaching and learning. This will be undertaken from a student 
engagement perspective, which encompasses the important roles lecturers play as 
mediators between students and institutions. Such understanding, it is argued, will 
provide solid grounds for effective staff development interventions. 


CURRENT SYSTEMIC INTERVENTIONS FOR STAFF DEVELOPMENT AND 
TRANSFORMATION 


Since 2008/2009 the DHET has implemented a dual institutional grant system to 
promote teaching and learning, and research. Following the recommendations of 
a national funding review, the two grants are being consolidated to form a single 
University Capacity Development Grant (UCDG). The consolidated grant will be 
guided by the University Capacity Development Programme (UCDP), which intends 
to intensify transformation goals at universities, while also enhancing quality, equity 
and success at universities (DHET 2017). The UCDP consists of the following three 
sub-programmes and activities, which aim to target current challenges impacting 
students, staff, and innovative programme development: 


Sub-programme 1: Student development 


Successfully guiding students through higher education remains a big challenge. 
While participation rates of students in higher education have increased substantially, 
prominent challenges such as low throughput/high dropout rates, achievement 
gaps, transitional issues amongst first-year students, and challenges first-generation 
students face are targeted through this sub-programme. Student development 
initiatives will focus on enhancing undergraduate and postgraduate student success 
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at institutions. This can most effectively be achieved by utilising evidence-based data 
as an indication to decision-makers on what activities are likely to yield the most 
benefit. Such high-impact activities include interventions such as first-year experience 
programmes, tutoring and mentoring programmes, academic support programmes 
as well as supplementary instruction programmes (see Chapter 5 in this book for 
more details about high-impact practices). Academic advising initiatives should then 
be designed to direct students to such activities to maximise their benefit. The students 
who are involved in these activities and who show great potential and interest in an 
academic career, should be included in early development opportunities. 


Sub-programme 2: Programme development 


The UCDG will assist universities with introducing new initiatives and programmes 
that have been identified as either a national priority and/or an institutional and 
regional priority in the higher education sector. This includes specifically piloting new 
technologies that should further staff development at universities. It is also important 
that the curriculum in South African institutions be transformed, and it is planned 
that funding will be made available for creating innovative curricula to address 


transformation requirements. 


Sub-programme 3: Staff/academic development 


Beyond the challenge of improving the status and quality of teaching mentioned 
earlier, challenges regarding academic staff include a skewed student-staff ratio, 
a lack of racial transformation in terms of staff appointments, too few academic 
staff with PhDs, and a large proportion of academic staff nearing retirement age. In 
recognition of these issues, in January 2015 the Staffing South Africa’s Universities 
Framework (SSAUF) was approved as part of the UCDG in an effort to develop 
professional staff at South African Universities (DHET 2017). As seen in Table 4.1, the 
SSAUF comprises of four academic and two support staff development programmes. 
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TABLE 4.1 Proposed programmes for the SSAUF 


SSAUF: Academic and staff development programmes 


Nurturing Emerging 
Scholars Programme 
(NESP) 


Aimed at developing senior undergraduate students and 
early postgraduate students (honours degree) who may be 
interested in an academic career. 


New Generation of 
Academics Programme 
(nGAP) 


Established to develop and recruit new academics that 
meet the set equity requirements and contribute to 
disciplinary areas that are required the most. 


Existing Staff Capacity 
Enhancement Programme 


(ESCEP) 


Designed to develop existing and newly appointed 
academics through developmental workshops, seed 
grants to continue with qualifications, teaching awards, 
and developing the scholarship of teaching and learning 
amongst others. 


Higher Education 
Leadership and 
Management Programme 
(HELMP) 


Supplementary Staff 
Employment Programme 


(SSEP) 


SSAUF: Supporting programmes 


Intended to develop university staff who are involved in 
leadership and management positions or are interested in 
these positions. 


Enables universities to recruit temporary staff with certain 
skills to assist with the implementation of the SSAUF 
academic and staff development programmes. 


Staffing South African’s 
Universities Development 
Programme 


Supports the core programmes with teaching and learning 
development, research development and professional 
development needs. 


(SSAU-DP) 


The New Generation of Academics Programme (nGAP) programme recruits annually 
and has involved the recruitment of over 100 new young academics in 2015, and 
an additional 75 new academics in 2016. An important transformational aspect of 
the programme is that the nGAP posts are permanent positions despite the target 
group (young black South Africans) being appointed at an earlier stage of academic 
development than has traditionally been the case. So, by way of illustration, the 
nGAP programme makes it possible for talented young scholars to be appointed with 
only a master’s degree, and to study towards a PhD. That the posts are permanent 
is a major incentive for young staff who often have onerous family responsibilities 
and cannot take the risk of a short-term contract post: this is a distinguishing feature 
of the nGAP This initiative is therefore responsive to most, if not all the challenges 
mentioned earlier. Along with the nGAP programme, and of relevance to the strong 
emphasis of the UCDG to incorporate evidence-based data relating to the areas 
of importance, is the focus on developing current lecturers to impact the quality 
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of student learning. This brings us to the focus of this chapter — to explore how 
data from the Lecturer Survey of Student Engagement (LSSE) can contribute to our 
understanding of what happens in teaching and learning contexts and where we can 
intervene to provide support and development. 


USING STUDENT ENGAGEMENT DATA TO FURTHER STAFF DEVELOPMENT 


Student engagement research is not only focused on students’ voices, but also the 
opinions and responses of the lecturers teaching today’s students. The LSSE was 
adapted from the Faculty Survey of Student Engagement (FSSE), widely used in the 
United States, and designed to accompany and complement the South African 
Survey of Student Engagement (SASSE). Because the UCDG requires institutional 
capacity development plans to be evidence based and data informed, the LSSE 
results could contribute significantly to evaluate grant objectives (DHET 2017). The 
LSSE measures lecturer expectations regarding students’ engagement in educational 
behaviours that are empirically linked with high levels of learning development. To 
gain insight into the ways in which lecturers think about student engagement, the 
LSSE also measures the degree to which they themselves are involved and invested 
in understanding student populations of undergraduate students at the university. 


To this end, the survey collects information on how lecturers spend their time on 
professional activities, such as teaching and research, and the kinds of learning 
experiences their institutions emphasise. It also provides valuable information on 
what staff think is important in teaching and learning, as well as information on how 
lecturers’ perspectives agree and disagree with those of students. The results can be 
used to identify areas of institutional strengths as well as aspects of the undergraduate 
experience that may warrant attention. The results can also be incorporated into 
student engagement results (from the SASSE) to create a greater understanding of the 
impact lecturer activities have on student learning and involvement. The information 
is intended to serve as a catalyst for discussions and developmental interventions 
in areas related to staff development programmes, scholarship of teaching and 
learning (SoTL) initiatives, assessment, institutional research, and curriculum reform. 
The following results from the 2014 LSSE administration demonstrate the contribution 
and implications of the data in relation to academic staff development. 


THE 2014 NATIONAL LSSE SAMPLE 


In the LSSE administration of 2014, 1 119 lecturers across eight higher education 
institutions completed the survey (response rate of approximately 16%). Most of the 
sample (57%) was from traditional universities, 11% from universities of technology, 
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17% from comprehensive universities, and 15% from private institutions. Of the 
sample, 44% were male and 56% female. The largest portion of the sample consisted 
of white individuals (59%) with 20% black African participants. The sample of black 
African staff members participating in the survey is slightly lower than black African 
academic staff representation at the eight universities during 2014, which ranged 
from 23% to 94% (DHET 2016). Staff were mostly Afrikaans (45%) and English- 
speaking (32%), and most of the lecturers were employed on a full-time basis (86%). 


In the LSSE 2014 survey the faculties were grouped into four categories: 


= Business, Economics and Management 
= Human and Social Sciences 
a Sciences, Engineering and Technology 


= Education 


Just over a third of the sample’s broad academic discipline falls in the category 
of Science, Engineering and Technology (36%) and another third in Human and 
Social Sciences (31%). Staff employed in the Business, Economics and Management 
discipline comprised 21% of the sample and staff from Education made up the 
smallest part of the sample at 12%. 


Staff were asked to indicate their academic rank, title or current position. Most of the 
sample indicated that they are employed at the level of lecturer (45%) and almost 
a quarter of the sample as senior lecturers (24%). The portion of the sample who 
are appointed as associate professors and professors make up 8% and 7% of the 
sample, respectively. Almost a third of the sample in this administration of LSSE was 
34 years of age or younger (31%) and 18% of the sample was above 55 years of age. 
Most of the staff reported that they have postgraduate degrees, with 36% indicating 
that they had attained doctoral degrees and 39% attained master’s degrees. When 
comparing lecturers’ age to their highest qualifications (Table 4.2), it is clear that 
the majority (41%) of the youngest cohort of 34 or younger had obtained master’s 
degrees, and 10% of the youngest cohort had obtained doctoral degrees. For the 
age group between 35 and 44, the majority of the lecturers’ highest qualification 
was either a doctoral degree (37%) or a master’s degree (44%). When looking at 
lecturers over the age of 45, the majority had obtained doctoral degrees. 
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TABLE 4.2 Comparison of lecturers’ age and highest qualification 


Age (years) 


Highest Ae 


qualification 35-44 45-54 55-64 |65 or older Total 
younger 


(Nu33s) | (N=285) | (N=266) | (N=171) | (N=20) | (N=1077) 

Doctoral 10% 37% 49% 61% 55% 36% 
Masters 41% 44% 38% 29% 40% 39% 
Honours 27% 13% 8% 5% 0% 15% 
ha a 7% 2% 2% 1% 5% 3% 
Postgraduate 
diploma or 3% 1% 1% 1% 0% 2% 
certificate 
ee first 2% 1% 0% 1% 0% 1% 
General 
academic 4% 1% 1% 1% 0% 2% 
bachelor 
ri al 3% 0% 1% 0% 0% 1% 
Other 3% 1% 0% 1% 0% 1% 

Total 100% 100% 100% 100% 100% 100% 


More than half of the 2014 national LSSE sample had been teaching for less than 
ten years. Staff with teaching experience of four years or less made up 27% and 
those with teaching experience of between five and nine years 26% of the sample. 
Another 26% of staff had been teaching for 10 to 19 years. The most experienced 
staff with 30 or more years of teaching experience made up the smallest portion of 
the sample at 6%. It could be argued that these lecturers already show some sense 
of engagement just by completing the survey, and therefore the results should be 
viewed in that light. It is also noted that the response rate, at 16%, was quite low 
across the eight universities. 


Lecturers were asked to think of an undergraduate module they had been teaching 
during 2014 and to indicate the amount of times they had taught this module prior 
to the current year. Most of the sample (30%) indicated that they had taught their 
selected module ten or more times before, and only 9% of the sample indicated that 
they had taught their module for the first time during 2014. 
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ANALYSING THE ENGAGEMENT RESULTS OF NEW AND EXISTING STAFF AT 
SOUTH AFRICAN UNIVERSITIES 


How do lecturers spend their time? 


Lecturers were asked to indicate how much time they spend in a typical seven-day 
week, specifically on teaching-related activities (Figure 4.1). The activities included: 
preparing class sessions, teaching undergraduate students, teaching postgraduate 
students, marking papers and exams, meeting with students outside of class, course 
administration (emailing students, maintaining course website, etc.), and working to 
improve their teaching (self-reflection, meeting with teaching consultants, attending 
teaching workshops, conducting research on their own modules/subjects, etc.). 


Time allocation: Teaching and learning 


| |, Ẹ 


Preparing Teaching Teaching Marking Merting with Admin Working to 
underaraduntes postgraduates students improve 
teaching 


TEIE 


FIGURE 4.1 Hours per seven-day week spent on teaching-related activities 


The largest amount of the 40.5 working hours related to teaching reported by 
lecturers is spent on teaching undergraduate students (8.6 hours or 21% of the 
time), which is consistent with expectations of staff at lecturer level. Thereafter, 
lecturers spend the most time preparing for their class sessions (7.8 hours or 19% 
of the time) and spend an average of 6.6 hours (16% of the time) marking papers 
and exams. Lecturers also report that they spend on average 4.9 hours (12% of the 
time) to improve their teaching through actions such as self-reflection, meeting with 
teaching consultants, attending teaching workshops, and conducting research on 
their own modules/subjects. Further, lecturers only put aside just over four hours per 
week to meet with students. 
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Further analysis was performed to examine the differences in the reported time 
spent of lecturers with different levels of teaching experience. For these purposes, 
the sample of lecturers were split into two categories, the first includes lecturers that 
have 0-4 years’ teaching experience, and the second group of lecturers have five 
years or more experience. Figure 4.2 shows the results of the analysis. 


Time allocation: Teaching and learning 


10 9.2 
8.5 8.5 
8 7.6 
6.4 6.6 

6 

4 

l J i iil 
0 


Hours 


Preparing Teaching Teaching Marking Meeting with Admin Working to 
uridergradusies postgraduates students improve 
teaching 


m0-4 years m5 years and more 


FIGURE 4.2 Hours per seven-day week spent on teaching-related activities — split by 
teaching experience 


The chart shows that the less experienced group of lecturers report an average 
work week related to teaching activities of 39.5 hours, while the more experienced 
lecturers report a 40.8 hour week. The less experienced lecturers spend more 
time preparing for class sessions (8.5 hours or 21% of the time) than the more 
experienced lecturers (7.6 hours or 19% of the time). The more experienced lecturers 
spend slightly less time (8.5 hours or 21% of the time) teaching undergraduate 
students than the less experienced lecturers (9.2 hours or 23% of the time); however, 
the more experienced lecturers soend more than twice as much time (3.9 hours or 
10% of the time) teaching postgraduate students than the less experienced lecturers 
(1.7 hours or 4% of the time). The two groups of lecturers spent similar amounts 
of time on marking papers and exams (6.4 hours and 6.6 hours respectively). The 
more experienced lecturers meet with students an average of 4.3 hours a week 
(or 11% of the time), compared to 3.5 hours (or 9% of the time) spent by the less 
experienced lecturers. 


Less experienced lecturers spend 4.8 hours of the week doing course-related 
administration, compared to 5.2 hours a week by the more experienced lectures. It 
seems that the less experienced staff spend more time on work related to improving 
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their teaching and research, with 5.4 hours a week (or 14% of their time), compared 
to 4.7 hours a week (or 11% of their time) spent by the more experienced lecturers. 


Minding the gap between lecturers’ and students’ perspectives 


Research points to the vital role that educators’ perceptions of their students have on 
the development of an equitable learning environment (Garcia & Guerra 2004). In 
the South African context in particular, Smit (2012) has argued that a persistent deficit 
thinking model among university staff towards students’ proposed lack of academic 
and cultural resources to meet the challenges posed by higher education, impedes 
students’ chances of success. Our findings from the LSSE, and in some respects 
from the SASSE, highlight a potential disconnect between lecturers’ assumptions and 
expectations of students, and students’ actual behaviours pertaining to academic 
and non-academic time allocation. We are mindful of the fact that using average 
scores across institutional types needs to be done with great care because contexts 
differ, however, the aim is more directed towards illustrating how staff and students 
are ‘missing each other’. The student sample used for the SASSE results below 
is from the same year’s administration — 2014. The SASSE data consist of nine 
institutions (one more than the LSSE, as universities can opt out of administering the 
LSSE with the SASSE). The majority of the 12 306 students from the 2014 sample 
were from four traditional universities, followed by three universities of technology, 
and two comprehensive universities. Similar to the lecturers’ faculty representation, 
most students were enrolled at faculties within the broader categories of Science, 
Engineering and Technology, followed by Business, Economics and Management, 
then Humanities, and the least were enrolled for Education. 


Regarding lecturers’ expectations of students’ time allocated to prepare for their 
identified modules, almost half (49%) of the lecturers expect their students to spend 
between three and six hours per week preparing for their module, while 50% of 
lecturers think that their students spend only one to two hours per week preparing 
for their module. In fact, 89% of lecturers report that they think their students spend 
less time preparing for class than the lecturers think is required. In addition to the 
negative perception of students’ commitment, the results also beg the question 
whether lecturers have a realistic view of students’ life demands. If the average 
lecturer expects students to spend an average of five hours per week preparing for 
their module, will students have enough time to prepare for all their modules, let 
alone all the other activities and responsibilities students need to spend time on? 
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Time spent 
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FIGURE 4.3 Comparison of lecturer estimations and student reports of time expenditure 


When comparing the LSSE and SASSE results, lecturers seem to greatly overestimate 
the amount of time students spend relaxing and socialising (Figure 4.3). First-year 
lecturers think their students spend 19 hours of a seven-day week relaxing and 
socialising, whereas the students report spending only nine hours. Similarly, senior 
lecturers think their students spend an average of 17 hours a week relaxing and 
socialising, but the students report spending only ten hours. Lecturers seem to correctly 
estimate the amount of time students spend traveling to class (seven hours) and doing 
community service (two hours), although slightly overestimating the time needed to 
provide care to dependents. Lecturers also overestimate the amount of time students 
spend working, either on or off campus. These results appear to confirm a negative 
view of students’ commitment to work in terms of an overestimation of relaxing and 
socialising, but does suggest an awareness of different demands on students’ time, 
although these tend to be overestimated when compared to student responses. It 
is important to emphasise that the point of this comparison is not who is accurate, 
lecturers or students, but rather to focus on which conversations are needed to effect 
greater alignment between lecturers and students. The data question how realistic 
lecturers are about the preparation demands on students across all their courses. 


Perspectives on student learning 


Questions related to student learning in the engagement surveys are based on 
Bloom’s taxonomy of educational learning objectives (Bloom, Engelhart, Furst, 
Hill & Krathwohl 1956). The framework provides classification statements of what 
lecturers expect of students, and what they intend for students to learn and is widely 
used in curriculum development and in academic staff development to support 
the development of outcomes and the alignment of assessment with outcomes 
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(Krathwohl 2002). The original goal of developing such a framework was to provide 
a common language across subject matters and different levels of students. Since 
its development, the framework has been revised and redesigned to include the 
following six dimensions: remember, understand, apply, analyse, evaluate, and 
create (Krathwohl 2002). 


To show how students and lecturers differ in their experiences of cognitive 
development and learning, we bring in student data from the SASSE to compare 
with lecturers’ responses. Lecturers were asked to indicate the degree to which their 
module drew on the cognitive functions as categorised in Bloom’s taxonomy. In 
the SASSE, students were in turn asked to indicate how much their academic work 
emphasised the same elements. Table 4.3 compares the percentages of lecturers 


and students who responded “Quite a bit” or “Very much”. 


TABLE 4.3 Comparison of Bloom’s taxonomy categorisation for lecturers and students 


Lecturer rating Student reporting 
Cognitive 


functions 


First-year Senior First-year Senior 


lecturers lecturers students students 
Memorising 58% 47% 75% 72% 
Applying 87% 92% 78% 80% 
Analysing 71% 77% 71% 73% 
Evaluating 61% 69% 71% 71% 
Creating 67% 71% 74% 74% 


The results suggest that there is an alignment between the emphasis that staff 
place on specific categories of Bloom’s taxonomy and the extent to which students 
experience this emphasis. First, it seems that students feel their work strongly 
emphasises memorisation within their subject areas (FY 75% and SNR 72%). In 
contrast, neither group of lecturers primarily teaching first-year students nor those 
teaching senior students feel that their selected modules place as much emphasis 
on memorising material (FY 58% and SNR 47%). This finding could also point to 
a misalignment between outcomes and assessments in that lecturers might state 
higher level outcomes in courses but these intentions are not realised in assessments, 
allowing students to pass through just using memorisation. A second concern with 
these findings relates to the lack of differentiation between first-year and senior 
curricula. These results suggest that lecturers do not actively emphasise Bloom’s 
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higher-order levels and therefore it is not surprising that the experience of first-year 


and senior students are so similar. 


Comparing lecturers with different levels of experience and their use of Bloom’s 
taxonomy, Figure 4.4 shows that lecturers with more teaching experience generally 
make more use of higher-level learning outcomes beyond memorisation. Looking 
at the most basic level of memorising material, 64% of the less experienced 
lecturers reported that this is emphasised in their module, compared to 47% of the 
more experienced lecturers. Fairly similar levels of emphasis are reported by less 
experienced lecturers and more experienced lecturers for applying facts, theories or 
methods (87% and 91% respectively), identifying different parts of an idea (74% and 
77% respectively), and evaluating a point of view, decision, or information source 
(65% and 68% respectively), with less experienced lecturers reporting slightly less 
emphasis on all three these activities. For forming a new idea or understanding, a 
larger proportion of the more experienced lecturers (72%) reported emphasis on this 
activity, compared to the less experienced lecturers (63%). A worrying interpretation 
points to the lack of emphasis lecturers place on having students evaluate information 
and forming a new idea or understanding — both of which are key elements of 
active learning. 


Emphasis on Bloom's taxonomy 
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FIGURE 4.4 Emphasis on Bloom’s taxonomy 


It is important to highlight that lecturers are also asked to indicate how important it 
is to them that students do certain activities in their specific module. In the SASSE, 
students are asked how often they do these activities during their current year 
at university. 
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Table 4.4 shows that lecturers place great importance on students needing to connect 
their learning to societal problems or issues, as well as including diverse perspectives 
in discussions or writing assignments. However, a minority of students report that they 
undertake these activities. 


TABLE 4.4  ltem-level comparison of lecturer expectations and students’ experience of activities 


FY SNR FY SNR 
lecturers | lecturers | students | students 


Connect his or her learning to societal 


À 84% 78% 47% 56% 
problems or issues 


Include diverse perspectives in module/ 


: : : re ; 68% 69% 42% 50% 
subject discussions or writing assignments 


Try to better understand someone else’s 
views by imagining how an issue looks 84% 77% 73% 71% 
from his or her point of view 


The LSSE asks lecturers how important they feel certain educational activities are for 
the students in their modules. Figure 4.5 shows these cognitive educational activities. 


While some differences might be attributed to disciplinary requirements or 
specifications, in general, both experienced and lesser experienced lecturers 
recognise the importance of cognitive educational activities to enhance engagement. 
The most important activities were learning something that changed the way students 
understand an issue or concept, and connecting ideas from their modules to their 
prior experiences and knowledge, with nearly 90% of lecturers prioritising these. 
Both groups of lecturers reported that it is important that students combine ideas 
from different modules when completing assignments (84% and 85% respectively 
of the less experienced and more experienced lecturers). The same proportion of 
less experienced lecturers and more experienced staff reported that it is important 
that students examine the strengths and weaknesses of their own views on a topic or 
issue, namely 78%. A slightly higher percentage of less experienced lecturers (77%) 
reported that it is important that students try to better understand someone else’s views 
by imagining how an issue looks from their point, compared to more experienced 
lecturers (72%). Both the less experienced and more experienced lecturers felt that 
the least important cognitive educational activity is including diverse perspectives in 
module discussions or writing assignments (60% and 57% respectively). 
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Beyond the learning outcomes primarily focused on classrooms, lecturers were 
also asked how important they felt it is that undergraduate students do certain 
activities, which could impact their learning experiences outside of the classroom 
before they graduate. The activities included: practical work related to their studies; 
holding a formal leadership position in a student organisation or group on campus; 
participating in student societies; developing an international perspective through 
campus initiatives and interacting with international students; working with a lecturer 
on a research project; participating in a community-based project (service learning) 
as part of a regular course; and developing a community project which requires 
them to use their university knowledge to address a problem in the community. 
These questions were rated on a 4-point scale (1 =Not important, 2=Somewhat 
important, 3=Important, and 4=Very important). The responses were recoded into 
“Not important” (option 1 and 2) and “Important” (option 3 and 4). Figure 4.6 
illustrates the results. 


Both the less experienced and more experienced lecturers felt that practical work is 
very important (85% and 80% respectively). The two lecturer groups felt that holding 
a formal leadership position was not important (70% and 76%). A larger proportion 
of less experienced lecturers (59%) felt that participating in student societies is 
important, compared to the more experienced lecturers (52%). Developing an 
international perspective is also more important to the less experienced lecturers 
(62%) than to the more experienced lecturers (55%). Fairly similar results are seen 
between the two groups for how important research with a lecturer is, with 67% of 
the less experienced lecturers and 63% of the more experienced lecturers rating 
this as important. Participating in a community-based project (service learning) 
is also more important to the less experienced lecturers (64%) than to the more 
experienced lecturers (59%). The largest difference between the two groups is seen 
for developing a community project, which requires students to use their university 
knowledge to address a problem in the community. A much larger proportion of 
the less experienced lecturers (68%) feel that this is important, compared to 56% 
of the more experienced lecturers. These results raise the question whether more 
experienced lecturers are less open to different and innovative ways of learning 
than the less experienced lecturers. Moreover, the general low enthusiasm for these 
co-curricular learning opportunities may also point to the ongoing process of slowly 
evolving institutional cultures which have not quite embraced the potential of co- 
curricular learning to help students develop and succeed. 


BUILDING ACADEMIC CAPACITY THROUGH STUDENT ENGAGEMENT 


EXPLORING TEACHING AND LEARNING PRACTICES 


Lecturers were asked to indicate what percentage of class time is used for certain 
activities. Table 4.5 is a conditional formatting visualisation to highlight most- 
frequently to least-frequently used techniques. Darker grey areas indicate the largest 
portion of the sample for each activity, and lighter grey areas indicate the smallest 
portion of the samples. 


TABLE 4.5 In-depth analysis of teaching and learning practice 


Activities 


Discussion 
activities 
Student 

presentation 

Assessing 
student 
learning 
activities 


Small-group 
Independent 
student work 
Movies and 
videos 
Experiential 


os 
aX 


18.2% 32.7% 30.1% 44.6% 40.6% 
Weve 31.9% | 24.3% | 35.3% EARO 


10.2% 23.3% 12.1% 


5.6% 10.8% 9.2% 


1.6% 7.5% 7.1% 


1.6% 47% 5.5% 


50-74% 0.7% 4.2% 5.7% 


75% or more 13.3% 1.5% 2.4% 1.5% 3.4% 0.4% 2.2% 


Total 100% 100% 100% 100% 100% 100% 100% 100% 


It seems that most class time is spent on lecturing (60-74%), followed by discussions 
(10-19%) and assessing student learning (1-9%). Most lecturers reported that they 
spend no or very little class time on independent student work, movies and videos 
and experiential activities. 


When splitting the sample into two groups based on years of experience, less than 
four years (Table 4.6) and five years or more (Table 4.7), not much difference 
emerges. The two groups both spend most of their time on lecturing (50-74%), 
followed by discussions (10-19%). However, the less experienced lecturers spend 
more time incorporating movies, videos, music or other performances than the more 
experienced lecturers. Furthermore, it appears that a larger proportion of the more 
experienced lecturers spend of their time assessing student learning than the less 


experienced lecturers. 
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TABLE 4.6 Teaching and learning practice — lecturers with less than four years’ experience 


Less experienced lecturers — Activities 


presentations | 


activities 
Assessing 
student 
learning 
activities 


2 
5 
(o) 
ee 
© 
E 
© 
£ 
n 


Independent | 
student work | 
Movies and 
videos 
Experiential 


21.5% 35.4% 34.7% 35.7% 9.4% 44.1% 


32.2% 


20-29% 6.0% 22.5% 13.2% 8.1% 9.5% 7.8% 13.5% 7.3% 


30-39% 9.6% 9.0% 7.9% 5.7% 5.8% 1.6% 7.8% 7.3% 


40-49% 18.5% 11.5% 4.1% 2.0% 5.4% 0.8% 5.3% 5.3% 
50-74% EEEE 6.6% 3.3% 3.3% 5.4% 1.2% 3.7% 2.4% 


75% or more 16.9% 2.5% 2.1% 2.0% 5.4% 1.2% 4.5% 2.4% 


Total 100% 100% 100% 100% 100% 100% 100% 100% 


TABLE 4.7 Teaching and learning practice — lecturers with more than five years’ experience 


More experienced lecturers - Activities 


Discussion 
Small-group | 
activities 
Student 
presentations | 
Independent 
student work 
Movies and 
videos 
Assessing 
student 
learning 
Experiential 
activities 


= 
al 
w 
& 
x 
w 
© 
Z 
z 


20-29% 7.4% | 22.1% 14.2% 8.1% 8.4% 4.9% 9.8% 9.8% 


30-39% 10.3% 10.5% 7.9% 6.4% 7.3% 1.5% 7.4% 7.0% 


40-49% | 23.8% 9.3% 4.2% 1.7% 5.8% 1.8% 4.5% 5.6% 


50-74% | 36.9% | 6.9% 4.4% 2.3% 5.2% 0.6% 4.3% 6.9% 


75% or more 12.0% 1.1% 2.5% 1.3% 2.7% 0.1% 1.4% 1.5% 


Total 100% 100% 100% 100% 100% 100% 100% 100% 
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DISCUSSION: IMPLICATIONS FOR ACADEMIC STAFF DEVELOPMENT WITHIN 
TEACHING AND LEARNING 


The data presented in this chapter raise several questions, and based on literature, 


some recommendations for academic staff development and working towards 


quality teaching and learning outcomes are proposed. These include: 


l; 


We know from student engagement literature that the relationship between 
lecturer and student is very important to enhance engagement and therefore 
impact learning and success. Further, as mentioned earlier, lecturers can play 
a vital role as role models, mentors and guides (Kuh et al 2010). The data, 
however, show us that lecturers only spend three to four hours per week meeting 
with students. In other words, maybe an hour per day from Monday to Thursday 
is set aside to meet with tens or even hundreds of students. While it is unfair to 
expect lecturers to be at the beck and call of students, serious discussions need 
to take place amongst lecturers, students and lecturer support staff to figure out 


ways of enhancing quality contact sessions between lecturers and students. 


The LSSE data show respondents spend around 40 hours a week on teaching 
and learning activities. This suggests that academic staff development initiatives 
will need to help staff manage their time through planning different parts of 
research projects at different times of the year. The data show that experienced 
teachers tend to value Bloom’s higher-level assessments more than less 
experienced staff, which might be the result of younger academics primarily 
teaching first-years. However, this still points to the need for staff development 
interventions that show young staff how to assess the higher levels of the 
taxonomy. Interestingly, more experienced staff appear to be less open to using 
innovative learning approaches, such as research projects, service learning, and 
using co-curricular spaces such as student societies. These results suggest that 
innovative approaches will need to be found for more experienced academics 


to help them see the value of these approaches. 


Lecturers indicate that they spend almost five hours per week on working to 
improve their teaching through actions such as self-reflection, meeting with 
teaching consultants, attending teaching workshops, and conducting research 
on their own courses, yet they still primarily make use of traditional lecturing, 
with very little incorporation of alternative ways of teaching. The data show that 
less experienced lecturers employ more of the alternative techniques and spend 
more time on working to enhance their teaching. These findings suggest that 
there might need to be an evaluation of how effective teaching and learning 
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development initiatives are at getting academics to reflect and change their 
teaching and learning approaches. 


4. Lecturers and students miss each other through expectations and actual 
behaviours. For example, almost 90% of lecturers completing the LSSE think 
their students do not spend as much time preparing for their lectures as is 
expected of them. Lecturers greatly overestimate the time students spend 
relaxing and socialising. Having high expectations of students is a key part of 
effective educational practices (Kuh 2007); however, these expectations need to 
be realistic. Having such data indicating where the gaps are between lecturers 
and students could prove invaluable for both parties to understand the point of 
view of the other. 


5. Regarding student learning outcomes or graduate attributes, the data show 
that lecturers find it important that students can apply the knowledge they 
gain; that they can connect knowledge to societal problems; that they are able 
to see others’ points of view; and that they will be able to evaluate, analyse, 
and personally relate to knowledge. That said, students seem to rely heavily 
on memorisation, and do not seem to be aware of much intentional focus to 
help them develop these skills. Adding to this gap is that most of teaching and 
learning time is spent on passively ‘transferring’ knowledge through traditional 
lectures. The problems here seem to be based on a lack of transferring intentions 
to practice; a lack of clear-cut educational outcomes known by all parties; and 
little evidence of efforts to incorporate different types of teaching and learning 
as well as assessment practices. 


6. In general, lecturers’ encouragement for students to take part in co-curricular 
learning opportunities is relatively low. This could be because these high-impact 
practices (see Chapter 5) are not normalised in institutions yet and thus people 
might not be aware of the potential benefits of these practices; or it might point 
to conservative views about where learning should take place. Either way, 
institutional cultures are often rigid and difficult to change. For large-scale co- 
curricular practices focused on enhancing student learning and development to 
be successful, lecturers’ support and encouragement are important. 


What lecturers want their students to learn requires active, collaborative learning. The 
use of learning management systems, the use of technology in classes, incorporating 
action research, making use of experiential learning and introducing alternative 
methods of assessment, and such like, are all important resources which could be 
used in response to the data. 
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As we have seen, student engagement data provide an important birds-eye view on 
what is going on within teaching and learning contexts. The LSSE offers a glimpse 
of lecturers’ time prioritisation, their beliefs about what and how students should 
learn, and how they put these into practice. Combining student responses from the 
SASSE with the LSSE enriches the data through providing an alternative view of how 
students spend their time, as well as their perceptions of how and what they are 
learning. The value of this combination is clear when we see how the experiences 
differ between students and lecturers. 


The results could be used as a warrant for the national staff recruitment and 
development projects mentioned earlier, especially in terms of informing the training 
and development of new staff in teaching and learning. Providing well-designed and 
effective teaching development opportunities at the beginning of academic careers 
should make a real difference both to the quality of teaching and learning, and 
to the confidence and effectiveness of new young staff members as they develop 
positive identities as academics. As has been recognised in the chapter, teaching is 
only one of the core components of the academic job, but has been the one that has 


arguably been the least systematically addressed and incentivised. 


For existing staff, the results also hold considerable value, as they do both a reality 
check on often unquestioned assumptions, and offer important insights into student 
experiences and insights. Drawing from an inter-institutional project to improve 
student engagement through academic staff development, Malnarich summarises 
common academic staff development practices emerging from the project from 


which we could learn: 


1. Pedagogical practices are introduced in relation to research on how people 
learn. Lecturers need to be encouraged to take the same scholarly approach to 
teaching and learning as they do in their own research fields. 


2. Existing good work is valued and made public. This can take form in institutional 
discussions or learning communities or through rewarding good practice, as 


well as more formal ways of sharing, for example, through SoTL publications. 


3. Student retention is addressed by focusing on the quality of students’ learning 
experiences. This suggests that integrative teaching and learning practices, as 
well as curricula which develop students holistically, need to be designed and 


implemented. 


4. What students are learning is examined in relation to expected learning outcomes. 


Achieving alignment in this regard requires using available data, such as different 
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forms of engagement data, to serve as catalysts for examined reflection and 
conversations about teaching effectiveness. 


5. Hard-to-solve dilemmas based on lecturers’ experiences of teaching are addressed. 
This finding suggests that organised sessions focusing on challenges lecturers face 
regarding student learning would provide a useful network of support. 


6. Work is assessed in relation to data and plans are revised. Relying on data 
should extend beyond the initial snapshot and should be used as a continuous 
method of evaluating progress (Malnarich 2008:4). 


It is important to note that the presentation of the data is not intended to trivialise or 
criticise the vital role lecturers currently play in helping students succeed — often with 
minimal support, tremendous tension between research and teaching workloads, 
and having to deal with the complexities of being mediators between students and 
institutions. It is exactly for these reasons that we believe stronger support systems 
need to be in place to help lecturers cope with the demands placed on them, as well 
as to optimally help students succeed. 


CONCLUDING THOUGHTS 


This chapter set out to explore how an evidence-based understanding of how lecturers 
think about and practice teaching and learning could impact staff development 
initiatives. On a national level, we see recognition for staff development and 
transformation initiatives in the form of strategic planning and funding allocation 
as illustrated in the University Capacity Development Grant. Through this chapter’s 
data contribution, we also see how important staff development initiatives need to go 
beyond merely focusing on equitable employment and encouraging improvement of 
academic qualifications amongst staff. 


The snapshot of what is happening in teaching and learning spaces provided by 
student engagement data offers a vital space for reflection, discussions, and SoTL 
initiatives; as well as developmental interventions. If effective teaching and education 
development support are heralded as the core business of an institution, significantly 
more effort needs to be focused on helping lecturers transfer their good intentions 
about what and how their students learn into practice. 


94 


BUILDING ACADEMIC CAPACITY THROUGH STUDENT ENGAGEMENT 


REFERENCES 


Bloom, B.S., Englehart, M.B., Furst, E. J., Hill, W.H. & Krathwohl, D.R. (1956). Taxonomy of 
educational objectives: The classification of educational goals. Handbook l: Cognitive 
domain. New York: McKay. 


Denney, F, Mitchell, A. & Wheat, K. (2015). Developing the next generation: Guidance and 
good practice in the leadership development of early career researchers and academics. 
Cambridge, UK: The Careers Research and Advisory Centre (CRAC) Limited. 


DHET (Department of Higher Education and Training). (2016). Statistics on postschool 
education and training in South Africa: 2014. Pretoria: DHET. Retrieved from http://www. 
dhet.gov.za/DHET%20Statistics%20Publication/Statistics%200n%20Post-School%20 
Education%20and%20Training%20in%20South%20Africa%2020 1 4.pdf 


DHET (Department of Higher Education and Training). (2017). Discussion document on the 
implementation of the university capacity development programme through effective 
management and utilisation of the university capacity development grant 2018-2020. 
Pretoria: DHET. 


Garcia, S. B. & Guerra, P L. (2004). Deconstructing deficit thinking: Working with educators 
to create more equitable learning environments. Education and Urban Society, 
36(2):150-168. https://doi.org/10.1177/0013124503261322 


Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory in Practice, 
41(4):212-218. https://doi.org/10.1207/s15430421tip4104 2 


Kuh, G.D. (2007). Director’s message. In: National Survey of Student Engagement (NSSE), 
Experiences that matter: Enhancing student learning and success. Annual report 2007. 
Retrieved from http://nsse.iub.edu/NSSE_2007_Annual_Report/docs/withhold/ 
NSSE_2007_Annual_ Report.pdf 


Kuh, G.D., Ikenberry, S.O., Jankowski, N.A., Cain, T.R., Ewell, PT., Hutchings, P & Kinzie, J. 
(2015). Using evidence of student learning to improve higher education. San Francisco: 
Jossey-Bass. 


Kuh, G.D., Kinzie, J., Schuh, J. & Whitt, E.J. (2010). Student success in college: Creating 
conditions that matter (2nd edition). San Francisco: Jossey-Bass. 


Malnarich, G. (2008). Increasing student engagement through faculty development: A practice 
brief based on BEAMS project outcomes. Retrieved from http://www.ihep.org/sites/default/ 
files/uploads/docs/pubs/beams_increasing student engagement _faculty_development.pdf 


Quinn, L. (Ed.). (2012). Re-imagining academic staff development: Spaces for disruption. 
Stellenbosch, South Africa: SUN PRESS. https://doi.org/10.18820/978 1920338879 


Scott, |. (2009). Academic development in South African higher education. In: E. Bitzer (Ed.), 
Higher education in South Africa: A scholarly look behind the scenes, 21-51. Stellenbosch, 
South Africa: SUN PRESS. https://doi.org/10.18820/978 19203381 83/02 


Smit, R. (2012). Towards a clearer understanding of student disadvantage in higher 
education: Problematising deficit thinking. Higher Education Research & Development, 
31(3):369-380. hitps://doi.org/10.1080/07294360.201 1.634383 


95 


DEVELOPING 
SOUTH AFRICAN 
HIGH-IMPACT 
PRACTICES 


Sonja Loots, Jillian Kinzie & Annél Oosthuysen 


An old story does not open the ear as a new one does — Beninese proverb 


INTRODUCTION 


The need for all postsecondary students to develop capacities, including broad 
knowledge, intellectual and practical skills, and personal responsibility to thrive in the 
21st century — and to develop these skills through involvement in high-quality learning 
experience in colleges and universities — has never been greater. The widespread 
focus in the United States on college completion through high-quality learning is 
central to both individual and societal success. A similar pattern is seen in South 
African higher education, where the national post-democratic focus of widening 
access to postsecondary education has expanded to include a strong emphasis on 
student success, particularly developing the skills necessary for economic growth, 


and fostering individual and social well-being. 


Fortunately, mounting evidence shows that particular educational practices are 
positively associated with the development of these skills and are correlated with 
important educational outcomes including transformative and deep learning and 
student success. Increasing opportunities for students to be involved in educational 
practices that make a significant impact on learning and success — dubbed “high- 
impact” because they result in desirable outcomes — are vital to help students get the 
most out of higher education. 


This chapter aims to unpack the conceptualisation and impact of high-impact 
practices (HIPs) implemented in the United States and explore the potential of 
developing and measuring HIPs to contribute to the development and success of 
students in the South African context. It should be noted that conversations around 
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HIPs in the South African context are still relatively new. The chapter therefore aims 
to contribute to these emerging conversations through providing a systemic view of 
the prevalence of HIPs as measured through the South African Survey of Student 
Engagement (SASSE). 


WHAT ARE HIGH-IMPACT PRACTICES AND WHAT DO THEY OFFER? 


The concept of student engagement does not only result from student effort, but 
simultaneously depends on whether the environment promotes academic and other 
areas of development. Practices with a high impact on student learning experiences 
and student engagement in general are not new concepts, however, the strength 
of the student engagement approach lies in the intentionality or purposetulness of 
aligning institutional strategies and practices to optimise the educational environment 
for student learning and development. 


The educational significance of ten promising high-impact practices, including 
first-year seminars and experiences; common intellectual experiences; learning 
communities; writing-intensive courses; collaborative assignments and projects; 
service learning; undergraduate research; study abroad and other experiences 
with diversity; internships; and capstone courses and projects, was first identified in 
the 2007 ground-breaking report, College Learning for the New Global Century, 
published by the Association of American Colleges and Universities (AAC&U). Many 
of these practices have enjoyed a long history in American higher education. First- 
year seminars, for example, are over 100 years old (University of South Carolina 
n.d.), yet recently, as evidence of their benefit to students from many backgrounds 
mounted, so did the advocacy for expanding their implementation. Arguing that 
involvement in quality learning experiences in higher education, not simply the 
receipt of a degree, makes a difference in terms of desirable outcomes, there was a 
strong call for more widespread investment in increasing students’ exposure to HIPs. 


In the decade since HIPs were declared demonstrably effective for fostering 
student success, there has been a surge of research on their effects and significant 
achievements in discerning what makes them effective and for which students (Finley 
& McNair 2013; Kuh 2008; Kuh & O’Donnell 2013). The official classification of 
a set of HIPs that make substantial contributions to student success was advanced 
by research conducted by the United States-based National Survey of Student 
Engagement (NSSE), which confirmed the strong positive effects associated with 
participation in six of the ten high-impact practices (NSSE 2007; Kuh 2008). HIP 
participation had a positive effect on students’ self-reported gains in learning and 
personal development outcomes, engaging in integrative approaches to learning, 
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and overall student engagement in effective educational practices. Notably, NSSE 
research demonstrated a compelling salutary benefit for students in groups that 
have been historically underrepresented in United States higher education — selected 
racial-ethnic populations and students with lower entering test scores. Brownell 
and Swaner (2010) further affirmed the value of five HIPs through their extensive 
review of published research on first-year seminars, learning communities, service 
learning, undergraduate research, and capstone courses and projects. Their report 
documented the deep research base demonstrating strong relationships between 
these HIPs and selected dimensions of student success, persistence, and changes 
in attitudes and behaviours. These studies helped establish HIPs as practices worth 
investing in for advancing student success. 


According to the Center for Community College Student Engagement (CCCSE) at 
the University of Texas, research has also expanded to identify and study a wide range 
of HIPs in community colleges, including a dozen practices, such as orientation, 
accelerated developmental education courses, and tutoring, that fall into three 
broad phases of students’ education, namely, planning for success, initiating success 
and sustaining success (CCCSE 2012). In general, studies have also found positive 
relationships between HIP participation and Grade Point Average scores (Huber 
2010) and developing a sense of citizenship (Kinzie 2012). Moreover, the gains in 
learning and personal development students report from their participation in HIPs 
relate to ‘deep approaches’ to learning, which, as opposed to surface-level learning, 
encompass a deeper understanding of the meaning attached to learning beyond 
mere memorisation of facts (Kuh 2008). Most importantly, is the persuasive evidence 
presented by Finley and McNair (2013) that students value what they experience in 
HIPs — real-world application for learning and caring relationships with faculty, staff 
and peers about substantive matters. 


It is this growing body of evidence on the developmental potential of HIPs which urges 
us to identify, implement, and measure the impact of HIPs in the South African context. 


DEVELOPING AND MEASURING SOUTH AFRICAN HIGH-IMPACT PRACTICES 
TO ENHANCE UNDERGRADUATE STUDENT ENGAGEMENT 


The University of the Free State’s Centre for Teaching and Learning student engagement 
research team foregrounded the development of context-specific HIPs in the 2013 
adapted version of the SASSE. At the time, the NSSE measured participation in six 
HIPs, four of which were also considered valuable scaled interventions in South 
Africa. These four HIPs were included in the 2014 administration of the SASSE: 
participation in practical work; in student societies; in service learning; and doing 
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research with staff. At the SASSE users’ workshops in 2015, discussions with the 
12 participating universities about HIPs revealed that most institutions felt that 
additional contextualised HIPs needed to be identified. Together with an extensive 
literature search, the SASSE project team consolidated these suggestions into a list of 
11 HIPs, which were included in the 2015 administration of the SASSE. 


Because SASSE administration follows a three-year cycle, most participating 
universities do not administer the surveys on an annual basis. Therefore, the 2015 
administration of the SASSE survey only included three higher education institutions, 
making it a good platform to pilot the proposed HIPs. The sample consisted of 
2 590 undergraduate students. 


The paragraphs that follow address each of the HIPs included in the survey along 
with literature supporting its inclusion, and present results (split between race and/ 
or generational status as proxies for historical/current underserved students) on 
students’ participation. The responses are shown as percentages and only include 
whether students indicated that they are planning to participate in these practices, or 
whether they have participated or are in progress of participating in these practices. 
Since the development of these programmes/interventions are in different stages 
and are considered differently in terms of importance, some of the proposed HIPs 
have been more researched than others. 


Participate in practical work related to studies (internship, work related 
learning, clinical placement, field experience, etc.) as a compulsory part of 
the degree 


Practical work related to study areas, such as internships, provide a valuable link 
between propositional knowledge and experiential knowledge, in other words, 
applying what has been learnt. Furthermore, gaining experience in the ‘real world’ 
promotes the development of generic and technical work-related skills valued by 
potential employers. A study on identifying graduate attributes from an employer’s 
point of view, commissioned by Higher Education South Africa and the South African 
Qualifications Association, found that “there is a real need to address gaps between 
employer expectations and higher education outcomes; this has largely to do with 
proactive task-directed engagement and the application of knowledge, i.e. the issue 
of knowledge-in-use” (Griessel & Parker 2009:20, italics added). Other studies 
supporting the need for practical knowledge include that of Meintjes and Niemann- 
Struweg (2011), who found that postgraduate students who took part in internships 
report increased professional development; expansion of knowledge and skills; 
greater interest in academic work; and relating theory to practice. Govender and 
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Taylor (2015), in developing a work-integrated learning partnership model within a 
human resources management programme, also report that all three role players 
(industry, academics and students) found such a partnership model beneficial for 


bridging theory and practice and developing skills valued in the workplace. 


Figure 5.1 shows participation in practical work by race! and generational status. Both 
racial groups shown as well as both groups representing generational status seem 
to recognise the importance of internships through their high indications of planning 
to participate in internships. However, white students seem to dominate actual 
participation in internships, and even their level of participation in this practice is less 
than 50%. Considering the importance placed on graduates developing practical 
work-related skills, as outlined above, the relationship between universities, students 
and the private sector needs to be strengthened to include more opportunities for 
students to develop these skills. 


Practical work participation 
100 
80 
xs 
E 59 55 56 
T 43 
© 40 
£ 24 28 29 
i a B ia 
0 
Black African White First-generation Non-first-generation 
Plan todo Œ Done or in progress 


FIGURE 5.1 Participation in practical work related to studies by race and generation status 


Participate in first-year experience seminars, including orientation and student 
learning groups 


First-year seminars and experiences usually take the form of structured or co- 


curricular programmes that bring small groups of new students together with staff 


1 Since racial category options for “Indian”, “coloured”, “mixed race”, “other”, and “prefer 
not to answer” were very small, these groups were excluded from the comparisons. Race is 
included as a proxy for historically underserved students, who have been found to benefit 
more from participation in HIPs in the United States context and is still relevant considering 
the persistent achievement gap in South Africa hovering at 8% between black African and 
white students (DHET 2017). 


100 


DEVELOPING SOUTH AFRICAN HIGH-IMPACT PRACTICES 


on a regular basis to help students transition to higher education and to engage in 
academic matters. The highest-quality first-year experiences and seminars emphasise 
critical inquiry, frequent writing, collaborative learning, and other skills that develop 
students’ intellectual and practical competencies, and can invite students to engage 
with cutting-edge questions in scholarship and with staff members’ own research. 
Other first-year experiences can simply include an extended orientation to the 
university supplemented by a course focused on introducing strategies for student 
success. Recognising the necessity to provide transitional and developmental support 
for first-year students, several scaled interventions have been put in place at different 
South African universities, such as curricular and/or co-curricular programmes, 
generic modules, or overarching First-Year Experience (FYE) programmes. The 
government-funded South African National Resource Centre (SANRC) for the First 
Year Experience and Students in Transition was established in 2014 and hosts an 
annual FYE conference. 


From first-year 2015 SASSE data (N=710), Figure 5.2 shows the levels of participation 
in FYEs between first-generation and non-first-generation students. Depending on 
the format of FYEs, these programmes or practices could play a significant role in 
helping students transition into higher education, particularly those who lack a sense 
of learning from others what to expect before accessing these institutions. While 
most students indicated that they are taking part in some form of FYE, this makes up 
less than half of first-year students. A worrying finding is the disinterest from non-first- 
generation students to participate in FYEs; however, it should also be noted that not 
all institutional programmes focusing on first-year support, development, transition, 
or success are marketed as a designated FYE. Even though there are descriptions 
accompanying SASSE questions, the lack of consensus in framing these interventions 
under an umbrella term might impact on how they understand the concept. 
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FIGURE 5.2 Participation of first-year students in first-year experience by generation status 


Register for an academic literacy or language development course 


The National Benchmark Tests (NBTs) were implemented in 2005. The tests were a 
result of a need to assess the linguistic and numeric literacy of learners completing 
high school. A recent report on almost 80 000 students taking these tests indicates 
that only 29% showed a level of academic literacy proficiency (scoring 64% and 
above) not requiring additional interventions or support (National Benchmark Test 
Project [NBTP] 2015). While academic literacy has been included as a stand-alone 
subject for struggling students as part of academic development, several authors 
have argued for the merging of academic literacy within faculties to promote deeper 
disciplinary content knowledge (e.g. Drennan 2015; Mgqwashu & Bengesai 2015; 
Naude & Bezuidenhout 2013). Registering for an academic literacy course is either 
compulsory for some, or may be considered irrelevant for those who do well in the 
NBTs. However, because of the need to develop academic literacy, this is considered 


a potential high-impact practice that deserves attention. 


Figure 5.3 shows students’ participation in literacy development courses. On average, 
only about 20% of the sample (irrespective of race or generational status) have 
participated in a literacy development course, while significantly less white and non- 
first-generation students indicate that they plan to enrol for such a course. Interestingly, 
also approximately 20% of students indicate that they have not decided whether 
they will enrol for a literacy development course. Considering the NBT numbers 
discussed earlier, these results raise questions about the preparedness of institutions 


to support and guide students adequately in promoting literacy development. 
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Participation in academic literacy 
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FIGURE 5.3 Participation in academic literacy or language development course by race and 
generation status 


Work with other students on a group project or assignment 


Levin (2005) contends that group work provides an educational learning opportunity 
for students as they are involved in the assessment and processing of alien values 
and ideas and react to unfamiliar knowledge territories. The value of interactive 
learning is also being recognised in South Africa. For example, McGhie and Du 
Preez (2015:177) argue that “peer interaction and cooperative learning not only 
strengthen the integration of students into the university environment, but they also 
improve their academic performance”. However, Monson and Redpath (2012) 
suggest that the potential of small group work to advance course objectives as 
well as contributing to navigate and appreciate diversity amongst students is still 
undervalued in South African higher education. They found that out of 71 third-year 
students, the most reported emotion associated with group work was frustration 
(42%), followed by enjoyment and excitement (34%), and camaraderie (32%). 
Questioning students about their experiences of group work, Thondhlana and 
Belluigi (2014) stress the importance of considerations such as intercultural learning 
and communication; ‘buying in’, motivation and attitudes of students; and having 
a common understanding between lecturers and students about the process and 
outcomes. In a similar exploration of student experiences with group work, Van 
Rheede van Oudtshoorn and Hay (2004) found that students associated group work 
with more active participation in class, and stress that the success of such modes of 
learning are dependent on commitment of individual group members paired with 
the lecturer’s effort to create a nurturing learning environment that supports creativity 
and challenges students to take ownership of their learning. 


Figure 5.4 shows students’ participation in group work by generational status. While 
the clear majority of all students indicate that they have participated in group work, non- 
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first-generation students show more participation. The importance of collaborative 
and peer learning is well documented in and beyond this book, particularly for the 
potential academic support such interactions with peers might bring. 


Participation in group work 
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FIGURE 5.4 Participation in group work by generation status 


Consult with an academic advisor to help with planning of studies 
and education 


In recent years, academic advising has been recognised as an essential, yet 
underdeveloped tool to promote student persistence, development, support and 
success. Although very few literature contributions have resulted from this recognition, 
the growing interest to develop academic advising should result in valuable academic 
insights in the near future. 


Figure 5.5 shows students’ participation in academic advising according to their 
indicated race and generational status. Black African students indicate higher planned 
and actual participation in academic advising than white students. Similarly, first- 
generation students indicate a higher score on planning to participate in academic 
advising, compared to non-first-generation students. However, non-first-generation 
students indicate a higher score in actual participation in academic advising than 
their counterparts. In general, barely more than a third (87%) of the 2015 sample 
participated in academic advising. 
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FIGURE 5.5 Participation in academic advising by race and generation status 


Make use of peer learning support (e.g. tutors, mentors, facilitators) 


The importance of peer learning support is evidenced through interventions 
responding to students’ low success rates. Naude and Bezuidenhout (2013) describe 
how pedagogical responsiveness in the form of facilitation scaffolding techniques 
significantly improved the marks of students enrolled in an extended curriculum, 
while even surpassing the average performance of students enrolled in a mainstream 
sociology module. Other authors, such as Davids (2014) and Layton (2015), have 
also argued that peer learning support should be at the forefront of experimenting 
with new pedagogical practices, which are more aligned with changing student 
populations. An example of such innovation is shared by Underhill and McDonald 
(2010), who developed a collaborative tutor development model through merging 
disciplinary and facilitation training in order to contribute to the transformative 
potential of tutorials. 


Figure 5.6 shows students’ participation in peer learning support by race and 
generational status. Black African students show higher levels of participation as 
well as much higher levels of planning to participate than white students. In contrast, 
non-first-generation students show higher levels of participation, but first-generation 
students show higher levels of planning to participate in peer learning. In general, 
over 60% of the sample participated in peer learning support. 
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Participation in peer learning support 
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FIGURE 5.6 Participation in peer learning support by race and generation status 


Explain module/subject material to other students as a tutor or learning 
facilitator 


Although literature on the benefits of tutorials in South African higher education 
institutions is growing steadily, very little is written about the benefits these 
programmes hold for those who do the tutoring. One such study examined the 
necessity of tutorials, particularly to impact the development of those entering 
university with disadvantages. Through training senior students to tutor struggling 
first-year students, the authors found that “the benefit was reciprocal as evident 
from the doubling of the number of distinctions in the second-year class from 1999 


(although subsequent student groups have exhibited a similar phenomenon)” (Page, 
Loots & Du Toit 2005:12). 


Figure 5.7 shows the racial and generational status breakdown of students’ 
participation in acting as peer tutors or learning facilitators. Black African students 
as well as first-generation students show a much higher intention to act as learning 
facilitators/tutors and act in these roles more than white and non-first-generation 
students respectively. 
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Explaining work as a tutor 
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FIGURE 5.7 Explaining work as a tutor by race and generation status 


Register for a mathematics or numeracy development course 


According to the World Economic Forum’s Global Competitiveness Report, South 
Africa ranks lowest out of 120 countries when comparing the quality of school-level 
maths and science education (Schwab 2015). Moreover, the NBT for Quantitative 
Reasoning shows that 72% of the test-taking population either need to enter an 
extended degree programme, or would need extensive and long-term support. 
A similar trend is seen with the NBT Mathematics test, of which the content is formally 
regarded as part of the secondary school mathematics curriculum. Only 11% of test 
takers showed a score proficient for mainstream higher education studies without 
additional support, while 16% would be likely to need additional support, and 73% 
are recommended to either attend extended programmes or receive intensive and 
longitudinal support (NBTP 2015). 


Figure 5.8 shows students’ participation in mathematics/numeracy development 
courses by race and generational status respectively. In general, only approximately 
20% of this sample were in the process of, or had completed a course in mathematical 
or numeracy development. Most white students in the sample were not planning to 
do such a course, whereas nearly a third of black African students also indicated 
this. Similarly, almost half of non-first-generation students and more than a third of 
first-generation students indicated that they are not planning to pursue mathematic/ 
numeracy development courses. When breaking down the results between faculties, 
it is not surprising that 41% of those studying in the Humanities and Social Sciences 
do not plan to pursue a course in mathematics/numeracy, while the two faculties 
most represented amongst those who are busy with such courses stem from the 
Science, Engineering and Technology (82%) and the Business, Economics and 
Management (27%) Faculties. 
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Register for a mathematics or numeracy course 
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FIGURE 5.8 Registered for mathematics/numeracy development by race and generation status 


Take part in service learning modules 


Studies identifying the developmental outcomes associated with service learning 
have found gains in personal development; interpersonal skills; learning to work with 
others in a group as well as independently; professional growth; increased gratitude; 
developing research skills; applying theoretical knowledge; developing critical 
thinking and reflection skills; innovation and creativity; problem-solving skills; gaining 
a sense of citizenship and social consciousness; and have found that developing, 
learning and understanding are facilitated through dialogue and collective reflection 
(Gaines-Hanks & Grayman 2009; Myburgh 2016; Naude 201 1; Pitso 2016; Stears 
2009). There has also been a considerable increase in theoretical contributions of 
engagement between higher education institutions and communities in recent years, 
thereby recognising the important role service learning and the broader concept of 
community engagement plays in reciprocal development (e.g. Bringle & Hatcher 
2005; Erasmus & Albertyn 2014; O'Brian 2005; Osman & Castle 2006). Recently, 
Myburgh (2016) and Pitso (2016) foregrounded the alignment of service learning 
curricula, and specifically its potential as a pedagogical practice, with preferred 
graduate attributes and employability skills favoured by a developing economy. The 
potential of service learning to develop students according to national prioritisation 
of higher education’s role in societal and economic development is clear (e.g. 
Department of Higher Education and Training [DHET] 2013). 


Figure 5.9 shows the prevalence of service learning modules in students’ courses 
by generational status. From the data, we see that first-generation students report 
more engagement with service learning projects within their modules than non-first- 
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generation students. In general, students report that more than half of their modules 
have some form of a service learning component. 


Participation in service learning 
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FIGURE 5.9 Participation in service learning by generational status 


Participate in student societies where students engage in topics related to their 
modules/subjects 


While some literature exists on the potential and successful implementation of different 
interpretations of learning communities (e.g. Harding & Engelbrecht 2015; Naude 
& Bezuidenhout 2015; Wilkinson 2014), very little, if any information regarding 
the impact of student-led disciplinary associations or learning groups is available. 
To participate in academic student societies relates closely to collaborative peer 
learning and doing group work, which have been positively associated with student 
development as indicated previously. It stands to reason that such societies could 
also provide significant social and academic support. 


Figure 5.10 shows the students’ responses on participation in student societies by 
race and generation status. Racially, there is a large difference between black African 
students’ intention to participate and actual participation in student societies when 
compared to white students who shows much higher disinterest in participation. 
While there is only a very slight difference in actual participation between non-first 
and first-generation students, the latter indicate a much higher intent to participate 
in such societies. 
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FIGURE 5.10 Participation in student societies by race and generation status 


Work with a lecturer on a research project as part of a module/subject, or as 
part of degree course 


There is a growing body of literature, particularly related to (participatory) action 
research projects, showing the significant transformative and developmental 
potential such projects could have for students. For example, Wilson-Strydom et al 
(2016) embarked on a three-year study with a group of 40 disadvantaged students 
to generate an in-depth account of students’ experiences within higher education as 
well as what helps them succeed and which challenges are just too much to bear. 
One tangible outcome of the study is a co-authored book between the researcher 
and all the students, where students could share their reflections on the project and 
their experiences. One student captures what this project meant: 

It allows me to understand the broad context of research and group 

work and interaction between the two. It also enhances my skills and 

development ... by exposing me to different topics and dismantling my 


stereotyped thoughts of being a law student who concerns himself with 
only the law (Wilson-Strydom et al 2016:67). 


Prioritising relationships between lecturers and students in this way has the potential 
to benefit students’ research and work skills development, as well as engaging in 
a more personal mentor-mentee relationship with lecturers. For these reasons, the 
SASSE includes working with a lecturer as a potential HIP 


Figure 5.11 shows students’ participation in projects with lecturers split into analyses 
of race and generational status. While slightly more non-first-generation students 
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participate in projects with lecturers, almost half of first- and non-first-generation 
students indicate that they plan to participate in such projects. 


Work with lecturer 
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FIGURE 5.11 Work with lecturer by generation status 


Taking a step back to look at overall participation of first-year and senior students in 
these HIPs, Figure 5.12 shows some promising findings. In general, there is relatively 
high participation in group work (68% and 72% respectively) and peer learning 
support (both 63%) among both first-year and senior students. Further, senior 
students indicate higher participation in most HIPs, implying a positive distribution 
of HIPs beyond the first year. This is important because of recent anecdotal concerns 
amongst lecturers about institutions placing too much emphasis and resources on 
first-year support while neglecting continuous support for senior students. Another 
promising finding is the slightly higher participation of first-year students in academic 
literacy programmes as well as higher participation in first-year seminars. As noted 
earlier — and will be expanded on later in the chapter — significant efforts on helping 
first-year students succeed have been put in place, the results of which might be 
starting to show here to some extent. 
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FIGURE 5.12 First-year and senior students’ participation in high-impact practices 


What is concerning from the findings is the general low level of participation in most 


of 


the HIPs measured. Without differentiating between year of study, the collective 


sample indicated that they still “Plan to do” the following HIPs: practical work 
(55%); participate in student societies (36%); work with a lecturer (44%); consult an 
academic advisor (36%); make use of peer learning support (22%); and act as a 
tutor or learning facilitator (40%). 


WHAT DO THE DATA TELL US ABOUT HIPs IN SOUTH AFRICA? 


Based on the evidence provided in the sections above, what we know thus far about 
HIPs in the South African context is as follows: 
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In general, and excluding participation in group work and making use of peer 
learning support, participation in HIPs is low. 


Participation in FYEs and academic advising (both approximately 40% of the 
whole sample) looks promising and national and/or institutional efforts to enhance 
support, particularly for first-years, seem to be taking off. 


A closer look needs to be taken into the literacy and numeracy abilities of students 
entering university and which scaled interventions could impact on providing 
support for those in need. 


Regarding racial differences, significantly more black African students plan to 
participate in the following HIPs: internships, FYEs, do more group work, consult 
academic advisors, make use of peer learning support, act as tutors, student 
societies, and projects with lecturers. While white students do not particularly stand 
out in their plans to take part in HIPs, they take part more than black African 
students in projects with lecturers, internships, registering for mathematics/ 
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numeracy course, and group work. White students also show more disinterest in 
participating in student societies and acting as tutors or learning facilitators. 

= First-generation students indicate high scores in planning to participate in HIPs, 
such as student societies, internships, FYEs, literacy courses, group work, academic 
advising, making use of peer learning support, and acting as tutors/learning 
facilitators. 

= More than half of the general sample report some service learning components in 
their courses. 

= The majority of students are either planning to do HIPs, not planning to do HIPs, or 
are unsure (as opposed to those actually participating). 


= HIPs need to be prioritised and scaled for maximum impact. 


However, the data overview presented here needs to be cautiously interpreted. 
First, discussions about which practices in the South African context could be 
considered to be of high impact are still ongoing. To date, there is not enough 
evidence on scaled practices showing a relationship with student development and 
success. In this regard, the items included in the SASSE give us an indication of 
whether students are participating or are willing to participate in practices where the 
developmental potential has either been proven, or where a developmental focal 
point is necessary. Of course, the inclusion of these items is always open for debate. 
A second consideration is that the inclusion, exclusion, or prioritisation of certain 
practices considered to be of ‘high impact’ might differ between institutions. This 
implies institutional consideration about which HIPs are aligned with their identities 
and priorities, as well as a commitment to enculture these practices as a vehicle 
for student success. Thirdly, there is a lack of an inclusive and common language 
of prioritised programmes or practices between staff and students. The lack of a 
common language might impact how students respond to SASSE questions. For 
example, while most first-year students would have participated in some version of an 
FYE, they might not link this experience to a ‘first-year experience’. Having a common 
language, for example, knowing what an FYE is, could contribute to ‘getting everyone 
on the same page’ in terms of expected participation, developmental outcomes, 
and should also impact the quality of the programme. A final consideration when 
interpreting these results is the small sample. As stated earlier, the smaller sample 
provided opportunity to pilot the inclusion of more HIPs; however, it should not 
be assumed that participation rates reported here are generalisable to the much 
larger public higher education sector. Sharing the data therefore has two purposes: 
(1) to stress the importance of institutional reflection in identifying, implementing 


and measuring scaled interventions targeted at student success; and (2) to provide 
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a snapshot of current participation rates in identified HIPs from a sample of around 
2 500 undergraduate students. 


THE WAY FORWARD 


Even though the developmental pathways of HIPs in the United States and South 
Africa are at very different stages, both still have to address issues of participation, 


scaling and determining the conditions for maximum impact on student success. 


Participation in HIPs at four-year colleges and universities in the United States 
ranges from half of all seniors reporting an internship, followed closely by about 
47% participating in a capstone experience, to the lowest proportion (only 15%) 
of seniors reporting a study abroad experience (NSSE 2015). Similarly, the SASSE 
data show that even though many students are participating in group work and 
peer learning support, participation in other HIPs are quite low. Considering that 
students’ willingness or plans to take part in HIPs outweigh their actual participation, 
we need to consider whether the opportunities for participation in HIPs are available 
to these students. 


The vast majority of the South African literature examples supporting the inclusion 
of the HIPs outlined above are reports on small-scale interventions, often limited to 
pedagogical contexts, which have made a positive impact on student success and 
development. Except for the national focus on supporting first-year students, none 
of these reported studies or interventions target institutional efforts to enculture high- 
impact practices on a large scale. A pressing need in the South African context is 
for institutions to identify and prioritise HIPs which they could take to scale. Similarly, 
in the United States, the idea of moving HIPs to the centre of the undergraduate 
experience, versus offering them as boutique programmes that only small proportions 
of students can experience has been a frequent call to ensure that all students can 
experience the benefits of HIPs (AAC&U 2007; Bass 2012). 


Kuh (2008) advocates ensuring two HIPs for all students — one early in the educational 
programme and one later in their studies. Students are also encouraged to engage in 
more engaging practices, because the more of these practices students experience, 
the more likely they are to reach key milestones. It is therefore critical to intentionally 
build many of these practices into every student’s educational experience (CCCSE 
2014). More recently, educators have made the case for requiring HIPs, increasing 
expectations to no fewer than three or four experiences in undergraduate education, 
and ensuring equitable participation in high-quality experiences. Some institutions 
have instituted graduation requirements that include participation in at least two out 
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of four sanctioned HIPs, while others have constructed elegant plans to increase HIPs 


by better integrating curricular and co-curricular experiences. 


Lastly, determining the conditions for maximum impact on student success relies 
heavily on combining existing knowledge with contextual research, knowledge and 
circumstances. In the United States, although there is plenty of evidence supporting 
HIPs, not all students necessarily gain the same developmental capabilities, nor to 
the same extent through HIP participation. Thus, further research on the impact of 
different HIPs on different students, as well as deeper analysis of specific aspects 
within broader HIPs, which contribute to development, is needed (Kuh 2007; 
Brownell & Swaner 2010). 


Another dimension of developing HIPs is tracking the extent to which students 
take advantage of them. Some HIPs like first-year seminars, service learning and 
internships have become more available across a range of institutions across the 
United States. Increasingly, community colleges have adopted learning communities, 
first-year experience programmes and internships (University of Texas at Austin n.d.). 
Interest in making HIPs more widespread has motivated some institutions to examine 
access to HIPs, while others have implemented initiatives to increase participation. 
For example, Indiana University-Purdue University Indianapolis (IUPUI) knit in HIPs to 
their educational requirements. IUPUI’s strategy to integrate undergraduate research 
or service learning into their required first-year learning communities ensured all 
students greater experiential learning and demonstrated a concerted effort to create 
HIPs reflective of their urban commuter student experience. The University of Wisconsin 
Madison devised a focused approach to introduce HIPs to new students beginning 
with orientation and throughout the year via advising. A curricular map, identifying 
where students could be exposed to HIPs in the undergraduate programme was made 
explicit to faculty and students. Hobart William Smith College focused on expanding 
opportunities for students to experience two HIPs, service learning and study abroad, 
that were most relevant to their mission. Convinced that these experiences made a 
difference in terms of persistence and engagement, administrators determined who 
was underrepresented in service and study abroad experiences (i.e., males and low- 
income students) and involved faculty and student affairs in devising approaches to 
making these experiences more possible for students. The contemporary focus on 
increasing student success and using evidence to improve have motivated campuses 
to advance the adoption of HIPs, document the educational benefits, and craft more 
effective approaches to support these practices. However, while research shows that 


participation benefits all students, not all students take part. It is time to seize the 
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opportunity to adopt intentionally structured curriculum for ensuring that HIPs are 
more widespread and available to all students. 


Turning back to identifying, developing and measuring HIPs in the South African 
context, the data we have on student participation raise serious questions on whether 
the environment is conducive to helping historically disadvantaged or underserved 
students move beyond a lack of awareness of which developmental initiatives they 
could benefit from and ‘planning to participate’ in HIPs, towards participating and 
benefiting from these practices. Further, the low participation in general means that a 
strong national and institutional intentional mind-shift is needed to identify, develop, 
and measure scaled programmes targeted at student success. This is particularly 
important because of the potential of HIPs to contribute to the additional development 
and support needed by disadvantaged students (Kuh 2008), which make up a large 
part of the South African student body. Additionally, such experiences add meaning 
to learning and contribute to how students shape their academic identities. “These 
are the kinds of experiences and transformative outcomes that are often used to 
characterise university education” (Coates & McCormick 2014:9). 


CONCLUSION 


Through this chapter we showed the developmental impact of HIPs in the United 
States context and how we are starting to foreground the importance of such scaled 
practices through the South African student engagement work. The HIPs identified 
and measured through the NSSE and SASSE surveys have been positively associated 
with the conditions or hallmarks which make HIPs successful. This in turn, impacts 
on student development, engagement and ultimately their successful completion 
of higher education. Although institutional values, missions and visions differ, it is 
safe to say that these are the core attributes which all higher education institutions 
should aim to pursue for their student bodies. Even though the potential of most of 
these HIPs have been brought into focus in South African context, there is a need to 
foster greater shared understanding about the value of making these practices more 
widespread. More research is needed on which scaled interventions have been put 
in place by institutions as well as evaluating the impact of such interventions. 


While the impact of HIPs in the United States context has not been questioned, 
concerns have been raised by educators involved in HIPs about the significant 
challenges associated with assuring quality, bringing these practices to scale 
and into the centre of the undergraduate education, and to ensuring equity in 
participation. The variety of programmes/interventions and the contexts they find 
themselves in makes it almost impossible to list best practices to ensuring quality, 
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however, rigorous planning and assessment, both through students’ perceptions of 
these experiences and through the direct evidence gathered from the products of 
student learning, should help build benchmarks of quality (Finley 2011). Finally, 
it is not surprising that the identification of an array of practices that when done 
well have a profound effect on student success has caught fire in postsecondary 
education, but it is only through the interest and dedication of higher education staff 
and broader institutional commitment that HIPs will realise their full potential for 
impacting student development, engagement and success. 
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PART I 


LEVERAGING STUDENT 
ENGAGEMENT EVIDENCE 
FOR INSTITUTIONAL CHANGE 
AND IMPROVEMENT 


Hamish Coates & Ali Radloff 


If you want to go quickly, go alone. If you want to go far, go together — 
African proverb 


INTRODUCTION 


This chapter is based on work by Coates, Radloff and Strydom (2014)! and Coates 
and Radloff (2014).? 


In recent years, student engagement has emerged has a priority for institutions. 
As post-secondary education continues to expand globally, keeping students 
engaged has become an issue for educators, institutional leaders and even for 
system level policy-makers. With greater choice than ever before, and increasing 
financial contributions to their own study options, students, prospective students and 
their families want to invest in an experience that is transformative and leads to 
successful outcomes. 


Student engagement spans considerable practical and analytic territory and opens 
myriad opportunities for investigation and educational improvement. However, a 
hard (if not the hardest) part of applied research is getting organisations and people 
to convert results into change. 


1 Used with the kind permission from the publishers: Coates, A. & Radloff, A. (2014). Broader 
strategies for developing student engagement. In: H. Coates & A.C. McCormick (Eds), 
Engaging university students: International insights from system-wide studies, 139-150. 
Singapore: Springer. 


2 Used with the kind permission from the publishers: Coates, H., Radloff, A. & Strydom, J.F. 
(2014). Role-based insights into enhancing student engagement. In: H. Coates & A.C. 
McCormick (Eds), Engaging university students: International insights from system-wide 
studies, 127-138. Singapore: Springer. 
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The first part of this chapter uses the role or designation as an international/ 
generalisable vehicle for engaging people in evidence-informed change. Taking a 
broader quality-improvement perspective, the second part of the chapter explores 
how institutions can use survey results to prompt change. Student engagement 
is a complex and dynamic phenomenon that often cuts across conventional 
organisational structures. Hence important challenges and opportunities are best 
considered from a higher level or even a cross-institutional perspective. 


Both parts of this chapter integrate broader perspectives from the insights that flow 
from involving thousands of people in large-scale projects over several years. Coates 
and McCormick (2014) provide an overview of these initiatives. The sections give 
life to strategic approaches for enhancing higher education by linking these with 
propositions for boosting student engagement. These are concept pieces that to 
work would need considerable refinement and adaptation within the organisational 


context and operational environment of a programme or institution. 


ROLE-BASED IMPROVEMENT INSIGHTS 


The deep and broad conceptual scope of student engagement weaves many 
stakeholders into enhancement work — institutional leaders, learners, lecturers, 
support staff, prospective students, parents and families. Large-scale studies offer 
much to many people. This breadth, at the same time, provokes the challenge of 
indeterminacy — of having too many change prospects on the table to know how to 
prioritise and proceed. Realistically, different people and organisations find different 
points of interest and relevance. Even given functional and labelling variations across 
institutions and systems, the role or designation offers a suitable means of presenting 
concrete suggestions for using concepts and evidence from student engagement 


research for enhancement work. 


These summary insights are filtered through several selected roles. The list is not 
intended to be exhaustive but rather to offer a canvass for distilling reflections from the 
cross-national studies. We explore the perspectives of quality assurance professionals, 
institutional researchers, programme coordinators, librarians, academic advisors, 
career development staff, and first-year experience and orientation programme staff. 
Chapter 8 explores the role of leaders in greater detail. 


Quality assurance professionals 


As earlier chapters of this book have sketched, the idea of measuring and monitoring 
student engagement took shape through research in the United States on how to 
provide the most pertinent data for institutional assessment. As a measure of what 
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students do, and how they are supported, engagement data provide important 
insights on key dimensions of institutional practice. These insights can stimulate 
enhancement and improvement initiatives that inform the quality assurance process. 


Quality assurance staff should ensure that student engagement is a part of institutional 
policy. Quality assurance staff have a persuasive influence on policy development 
about the institution’s quality objectives, and the principles and processes that 
will underpin monitoring and review. Given the relatively recent development of 
student engagement, several institutional policies may need to be revised to take 
student engagement into account. Processes for organisational review and annual 
monitoring of and reporting about quality, may need ongoing adjustment to ensure 
that facets of engagement pertinent to a particular institution are considered. 


Annual monitoring processes can require a close attention to student engagement 
data, or to specific aspects of the phenomenon. Policy may require that improvement 
plans resulting from organisational reviews must have specific reference to student 
engagement practices within the unit under review. 


Usually, improvement plans are targeted at lifting performance in an organisational 
unit. It can also be worth building into policy that good performance in one unit will 
be explored further and reported to other units. For example, data may show that 
part-time students across an institution are unhappy with the learning skills support 
they receive, except for those part-time students in the school of business at a regional 
campus. Conducting focus group interviews and staff surveys or documenting the 
practice of advisers, could be ways in which to extend an understanding of good 
practice as reported by students, and to promote adoption of that practice elsewhere 


within the university. 


Profiling good student engagement practice can be used as a helpful reference 
point. Quality assurance staff in universities have the privilege open to few others — 
of being familiar with good practice across their institutions. There is no one better 
placed than quality assurance staff to profile good student engagement practice. 
Building the student engagement good practice network can be a key part of their 
role in prompting innovative solutions and responses to the student experience. 


For example, results from student surveys, supplemented by some additional research, 
may reveal that the way in which tutors in a first-year engineering subject have linked 
tutorial groups via a wiki has had a remarkable impact on student engagement. 
Tutors in a first-year linguistics subject, who are struggling with student engagement, 
may never hear about the engineers’ wiki, unless they are informed of the successful 
practice in engineering. Systematically profiling good student engagement practice, 
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across institutions, across campuses, across faculties and departments, is a key 
element of a quality assurance role, including informing faculties and departments 
of the existence of such a system. 


Institutional researchers 


Institutional researchers play a vital role in analysing, interpreting and communicating 
insights on student engagement. Results provide a variety of staff and students with 
a wealth of information, and can inform course reviews, support strategic analysis at 
the academic staff and campus levels, and enrich reviews of support services offered 
by administrative divisions. 


A considered approach to using data on students’ engagement ensures that the data 
and analyses are fully understood so that the greatest benefit from collecting data is 
achieved. Such an approach includes reporting results to staff and to students, and 
on actions planned by the institution in response to survey outcomes. Institutional 
researchers can therefore demystify student engagement data. Users of data need 
clear information and advice about what the data and analysis do — or do not — 
tell them. They can offer their colleagues a comprehensive understanding of the 
different facets of student engagement. They can explain the methodologies used to 
develop the survey sample and to determine results. 


Institutional researchers can also identify and respond to the data needs of different 
audiences. Reports and analyses need to be fashioned and presented so that they 
meet the needs and interests of different audiences. A summary university level report 
will offer a level of analysis and detail different from that provided for a course 
review. Similarly, a summary report for students of empirical insights into their own 
engagement demands a particular focus. 


Responding to the specific needs of different audiences requires that summary tables 
and charts be supported by textual explanations to ensure understanding of what 
is being presented. Institutional researchers may provide support and advice to the 
interpretation of evidence on engagement by advising reference/steering groups, 
liaising with academic development units, and contributing to resources developed 
to enhance the student experience. 


Of course, engagement data should be interpreted within an institutional context. 
Maximum benefit from data is gained through robust discussion of student 
engagement that is informed by data analysed and interpreted within the context of 
an individual institution. What is important to each institution at a given point in time 
will depend on contextual factors relevant at that time. Interpretation of the data must 
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recognise these contextual factors. Institutional researchers can undertake advanced 

analysis of student engagement data. They can support the nuanced application of 

data in many ways, including: 

# benchmarking at the institution level, or at a lower level of aggregation (e.g. faculty, 
school, course), depending on response rates and number of responses; 

= linking with other surveys, enrolment and assessment data sets; and 


"analysing open text comments (see Chambers & Chiang 2012) to make best use 
of this valuable feedback on student engagement. 


Benchmarking is an activity that institutional researchers and their teams have the 
skills and knowledge to support. Productive benchmarking depends on nuanced 
interrogation of data. Comparisons within and between institutions must account 
for variations in cohort composition, such as relative proportions of part-time and 
full-time students, or of international and domestic students. These variations have a 
significant influence on an institution's student engagement outcomes. 


Department chairs 


Department chairs play a significant role and are a pivotal in shaping successful 
student engagement. They have a definitive influence over how student engagement 
is integrated into programme design and how it is reflected in teaching practice. They 
influence how the programme actively links its students to learning skills support, 
student services and the wider life of the university. They influence the development 
of policy and processes so that they promote high-quality learning outcomes for 
their students. 


Department chairs can put student engagement on the agenda when a programme 
is reviewed. Programme and course or unit reviews are good opportunities for 
shaping curriculum in ways that promote student engagement. Because so much 
learning goes on outside the classroom, a key area for focus may be to influence the 
kind of learning that takes place outside the lecture, the tutorial or the laboratory. 


It is important to support teachers to work with student engagement principles. 
Tutorials and laboratory sessions often rely on a variety of part-time and adjunct 
teachers who may have limited knowledge of the principles that underpin student 
engagement. Yet teachers are the academic staff with whom most students have 
the greatest level of interaction. Department chairs can ensure that induction 
and professional learning opportunities for teachers include reference to student 
engagement research and to the way in which student engagement principles are 
embedded in unit learning and assessment designs. They can ensure that teachers 
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are introduced to ideas about how to promote student engagement in small group 


learning environments. 


Tutorials provide ideal opportunities for structured peer-learning activities, which have 
additional benefits beyond the tutorial. Properly managed, they can foster beyond- 
classroom study groups, friendships and informal networks by bringing together 
students who may not know each other. As the principles sketched in the opening 
chapter and country case studies affirm, both peer learning and connectedness 
increase student engagement. 


Department chairs should identity opportunities for student-staff interaction beyond 
the classroom. Creating opportunities for interaction between teachers and students 
is especially important for sustaining student engagement. Students value formal 
and informal interactions outside the classroom. Such interactions can reinforce 
motivation, a sense of purpose, a sense of connection. As the literature and findings 
in the earlier chapters convey, they contribute to student persistence and better 
academic performance. 


Part of the challenge is to establish opportunities for frequent interaction with teachers. 
Inherent in the conceptual and empirical frames that shape student engagement is 
an understanding that student-staff interactions have a significant impact on high- 
quality learning. In the Australian case, as with the South African data presented in 
Chapter 2, it is disconcerting that so few students report meaningful and frequent 
interactions with lecturers outside the classroom (see Radloff & Coates 2014). 
Australian teachers have seen this differently, with results from the academic staff 
surveys showing that more staff members than students believe they have meaningful 
and frequent interactions with students beyond the classroom. Exploring these 
perspectives with students and staff may clarify what is at the heart of such divergent 
views and provide a platform for insightful change. 


Librarians 


Librarians offer a particularly useful lens for exploring the leadership of student 
engagement. Libraries share much across organisational and national borders, and 
have tended to be early adopters and advocates of enhancement practices. 


Libraries can engage students with learning in many ways. The physical space itself 
can be used to ensure that there are sufficient resources available to students, enough 
independent study areas, collaborative learning spaces and sufficient numbers of 
computers or wireless access areas. Although increasing numbers of students are 
not campus-based, it is still important to consider how the physical library space can 
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best engage students with the library and with learning. Are the collections easy to 
access? Do the operating hours meet students’ needs? Are there enough computers 
or areas where students can connect to the internet wirelessly? Are the silent study 
areas and collaborative learning spaces adequate? How could they be improved? 


Libraries also exist in the virtual world, allowing distributed learning to take place, 
and allowing access for external, distance and online students, and for more flexible 
kinds of learning. Librarians contribute to student engagement by providing expertise 
to students through formal workshops or tutorials and by providing guidance 
informally. Librarians help students learn the best ways to access and use quality 
information and resources, help them to enhance their study and research skills and 
explain how to use the latest technologies to enhance their learning. It is important to 
increase student access to library resources as much as possible. Increasingly, even 
at campus-based institutions, students are studying part-time, online, externally or 
by distance. As a result, university libraries are progressively becoming distributed 
learning spaces to cater for more flexible learning, and for part-time, distance and 
external students. 


Many university libraries now use social networking sites to keep their students up- 
to-date on new acquisitions, upcoming workshops and classes, and changes to 
opening hours. Students who add their university library to social network accounts 
receive these updates as part of their newsfeed. Other universities use feeds or send 
out targeted emails to students to keep them updated on what is happening at their 
library. Students are now used to accessing library resources via the web at any time. 
Universities are making access to library resources even easier and more flexible by 
allowing students to add their catalogue searches to personalised homepages, or 
by letting them search the catalogue using social networking sites. Some universities 
have even created a mobile web version of their library website and catalogue, 
giving students the flexibility to access their library on smart devices, at whatever time 
and wherever they are. 


Libraries also make it easier for students who cannot come on campus to access 
librarians and their knowledge. Libraries do this in various ways: through online 
functionality, such as live chats, instant messaging or online forums where they can 
consult with library staff, and through podcasts of classes or tips on researching, 
referencing and using library resources. 


First-year experience coordinators 


As indicated in the cases of the United States, Canada and South Africa, the data 
obtained from the range of student engagement surveys can be meaningfully linked 
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over time at the individual student level to understand how students engage. The 
particular example of linking the Beginning College Survey of Student Engagement 
(BCSSE) (Beginning University Survey of Student Engagement [BUSSE]) and the 
National Survey of Student Engagement (NSSE) (South African Survey of Student 
Engagement [SASSE]) data to inform first-year experience and orientation programmes 
has been noted in research (Kuh, Kinzie, Schuh & Whitt 2010; Strydom, Oosthuysen, 
Hen-Boisen, Loots & Posthumus 201 7). 


First-year experience (FYE) and orientation programmes serve as an avenue where 
student engagement data can be used to design and monitor curricular experiences 
for first-year students, while simultaneously being used as content material. Using 
matched engagement results such as this in the design of FYE curriculum presents a 
unique opportunity for a campus to require students to participate in a wide range 


of effective educational practices. 


Using data from recent engagement survey administrations in the content material 
of such programmes is a powerful way of bringing research results ‘closer to home’ 
and making students aware of the experiences of their peers. Furthermore, by 
incorporating research results into programmes, students are made aware right from 
the start of their campus experience that the institution takes assessment seriously and 
that their input in research processes is not only valuable, but also used meaningfully 


by campus leaders. 


Academic advisors 


Academic advisors help students develop their capacity to participate effectively in 
learning by helping with sound education planning and referring them to resources 
that help them with academic success skills. In essence, academic advisors play a 
role in providing support given their student-focused perspective. Their contribution 
to student engagement may be direct, through the teaching of academic skills, or it 
may be indirect, through a referral to other services that promote wider aspects of 
student engagement. Knowing about the range of services and other avenues that 
support student engagement is an important part of every advisor’s role. 


Advisors may work one-on-one with students, supporting them to develop strategies 
to more actively contribute to tutorial discussion: asking questions, critically 
analysing information, and volunteering answers. They can help students understand 
why attending lectures and reading materials before the class and/or tutorial are 
key to effective participation. They might draw the attention of a shy or lonely first- 


year student to university clubs and societies. Advisors are able to identity students 
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volunteering opportunities that may help them develop confidence, make friends 
and feel more engaged with the university community. 


Advisors can design language support programmes to build student engagement. 
They can play an important role in student engagement by creating, and helping 
to maintain, support programmes that extend beyond the traditional classroom or 
advisor relationship. One possibility would be working with community agencies 
to integrate a volunteer component into language support programmes. Through 
community involvement, students can practice communication skills, develop wider 
personal networks and contribute directly to the university’s engagement with its 
region. Another example would be to design and support, perhaps in concert with a 
faculty or academic programme team, a conversation club for international students 
that enlists domestic students as facilitators. Advisors could provide training to the 
domestic students on how to lead a group effectively. This kind of exchange can be 
both fun and rewarding. International students can make connections with domestic 
students and learn more about the local culture. Strategies like this would likely yield 
benefits for domestic students too as they become more engaged in university life 
through contact. 


Advisors can put their advisory skills to work in clubs and societies. They have a 
range of capabilities that can directly support student engagement and academic 
skills development. They can connect their capabilities to the needs of clubs and 
societies and other student services. For example, if a student society or club is 
planning an event, advisors could contribute by using the planning phase as an 
opportunity to teach teamwork and project management skills. 


Careers advisors 


A significant number of students’ academic engagement is undertaken to prepare 
for life after higher education. Employers recruit graduates who are well rounded 
and have a range of skills over and above those gained in the classroom. Careers 
advisors can encourage students’ involvement in a variety of extra-curricular activities, 
both at university and externally which will develop their skills in a range of key areas. 
Extra-curricular activities help students to develop the broader skills that employers 
seek. These skills are often related to universities’ graduate attributes and include 


communication, teamwork, problem-solving and organisation, among others. 


Participating in extra-curricular activities enhances students’ résumés and interview 
content. Students can present evidence of participation in these activities to 
demonstrate they are well rounded and have a range of sought-after skills. In 
addition, it seems likely that students’ knowledge of themselves — their interests, 
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aptitudes, values and strengths — grows enormously with exposure to a range of 
different activities and experiences. This, in turn, makes for more informed career 
decision-making. 


Consider the best ways for students to get involved. On campus, students can host 
new students during orientation, become a peer mentor, assist on open days, write 
for the student newspaper, serve as an office bearer for the student union, and be 
actively involved in a club or society. Off campus, students can do volunteer work, 
which is especially useful if the work is related to the student’s career goals. Other 
activities that could be valuable in broadening students’ lives and their development 
beyond the university include sporting clubs, community groups, involvement in a 
range of interest areas, and of course, part-time or casual paid work. These activities 
develop skills and knowledge and enable students to learn more about themselves 


and their potential. 


A key role of the careers professional is to help students understand the value of 
participating in these activities, and to provide students with insights in the learning 
that can occur through involvement in them. Assisting them to see the relevance 
to later employment and where careers may link in with these activities, is one of 
the main responsibilities of the careers professional. Careers advisors could hold a 
volunteer day or a careers fair, where they can advertise opportunities, such as hosting 
or mentoring students and volunteering for various organisations. Some strategies 
for careers staff include emphasising the benefits of getting involved in volunteering 
and other activities — not only to students but also to academic colleagues. This 
could include preparing a handout in orientation material about the value of getting 
involved, and making sure casual and part-time work advertisements highlight why 
this type of work could benefit a future career. 


Even in the most elite higher education settings, work-integrated learning is one of 

the key aspects of student engagement that is relevant to staff involved in student 

careers development. Work-integrated learning provides many benefits to students, 

and gives them: 

= an opportunity to put theory into practice and gain valuable workplace knowledge 
and skills; 

= an insight into an industry or career they may be interested in pursuing; 


= a range of valuable experiences and insights that can be highlighted in the résumé 
and during interviews; and 


= better insight into, and appreciation of, theoretical content on return to the 
classroom. 
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Careers staff can best assist students to extract the most from work-integrated 


learning by: 


assisting students to best understand what they have learned and experienced, and 
why that is valuable to employers; 

assisting students to optimally present both the skills and knowledge they have 
gained via their résumé; and 

encouraging students to document their experiences in an e-portfolio when that 
option is available. 


For other stakeholders, careers staff can also add to the value of work-integrated 


learning by: 


working with academics to ensure they are assisting students to understand the 
value of work-integrated learning; 


encouraging students who have undertaken work experience or another form of 
work-integrated learning to speak to other students in class to convey the benefits 
and give other students advice; and 

determining if a graduate employer may be prepared to take students on for a 
period of work experience and linking with employers to see if they could take more 
students and students from other areas. 


In summary, to enhance student engagement, careers guidance staff can: 


act as a key source of expert, current knowledge in the university about what 
employers look for in the way of skills and knowledge in addition to technical 
knowledge and classroom learning; 

collect resources relevant to these different forms of knowledge — directly or 
indirectly — through the campus, community, etc., and interpret information for 
the students; 

make these resources available and visible to students; 

promote the value of involvement to students (and academic staff) at every 
opportunity; 

publish ‘good news’ stories of student endeavours wherever possible, using 
websites, university newspaper and flyers; and 

ensure workshops and individual consultations with students highlight the 
importance of, and possibilities for, student involvement with relevant groups on 
and off campus. 


Clearly, there are myriad ways in which careers staff can encourage and help 


students participate in activities that enhance learning and development outcomes. 


This is important in and of itself. Encouraging students to participate in these types 
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of activities influences students augment capabilities that will help them build their 


future careers. 


BROADER STRATEGIES FOR DEVELOPING STUDENT ENGAGEMENT 


As more people participate in higher education globally, and as new technologies 
spur new forms of delivery, the nature of education becomes more complex. For 
institutions, engaging students can mean more than just making sure they don’t drop 
out. It can create long-term and strategic opportunities including attracting prospective 
students, developing graduate attributes such as team-work and communication 
skills, producing highly motivated graduates, establishing meaningful alumni 
relationships leading to mentorships, enhancing staff development and satisfaction, 
developing innovations, and enhancing institutional reputation. 


With the intention of being helpfully prescriptive, the following analyses review 
development of institution-wide approaches to enhancing student engagement. 
Rather than work through the role, the following sections unpack several inter- 
related improvement strategies. The strategies cover developing institution-wide 
approaches, establishing benchmarking for continuous improvement, broadening 
staff involvement in student learning, enhancing student interactions with staff, and 
monitoring quality data over time. 


Developing institution-wide approaches to student engagement 


Responding to student needs is a basic tenet for engaging students. Developing a 
successful student engagement strategy means building student perspectives into 
the way an institution organises itself around students. The starting point is students 
rather than institutional structures and procedures. 


Student engagement rests, first, on the idea that students are independent learners 
with responsibility for managing their own education. Second, it rests on the concept 
that a university has a responsibility to create an environment that supports and 
encourages students to manage their learning effectively. The selection of student 
engagement strategies is informed by considering how to allocate institutional 
resources so that creative and productive relationships are established between 
students, their learning journeys and the institution. 


Akey characteristic of effective student engagement is an integrated web of supportive 
institutional practices. Establishing and maintaining that web involves identifying the 
potential for creative and productive links between practices as diverse as career and 
employment services, student guilds, advising, learning support, study abroad, peer 
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tutoring, department academic strategies, the use of diagnostic assessment designs 
in the early part of semester, cafeteria hours, and so on. 


It is vital to develop responsive strategies for effective student engagement. The 
selection and development of strategies that promote engagement will depend to a 
considerable extent on: 


= academic and professional staff understanding the student engagement evidence 
base and theoretical contributions that have shaped the concept over the years; 

= close appreciation of data, including student engagement data, that will help guide 
the crafting of strategies; 

= recognition that students are not homogenous, and that different strategies will be 
needed for different groups of students; and 

"recognition that institution-wide strategies cannot substitute for student engagement 
strategies at the academic department level. 


It is particularly important to understand the principles of student engagement. 
Student engagement has a deep evidence base built up over several decades. 
Academic and professional staff are more likely to accept that strategies for student 
engagement should have some priority if they understand the evidence base for 
student engagement and the principles that follow from it. This matters because 
all staff are busy and all have multiple priorities and commitments. Adopting new 
strategies means that existing workloads and priorities may have to be reconsidered 
to make room for those strategies. Sometimes adopting new student engagement 
strategies will require incremental changes to existing practices. At other times, new 
strategies will require more than adaptive change — they will require adoption of new 
ways of working as well as new approaches to recognising and rewarding that work. 


An understanding of the principles of student engagement can act as a filter and 
a guide for academic and professional staff who are planning and reviewing, or 
making minor adjustments to, academic programmes, assessment practice, teaching 
designs, student learning and other support services, orientation programmes, open 
days, career services, or approaches to supporting peer-to-peer interactions in both 


academic and social contexts. 


The best means of securing a deeper understanding of student engagement will 
differ between and within universities. Specific professional development on student 
engagement may be necessary. It may assist if other professional development 
activities incorporate student engagement perspectives. Task-specific support may 
help. Targeted and knowledgeable input on student engagement could be offered 
to a teaching team refreshing first-year engineering subjects, a group charged with 
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developing a capstone subject in international development, or course advisors 
evaluating the effectiveness of their service delivery over the past year. 


Analysis and interpretation of data will be instructive and influential if the principles 
of student engagement are widely shared. The data can be a key input to selecting 
strategies, and to their planning, monitoring and evaluation. Particularly when 
compiled from multiple sources and modes, it can provide a firm foundation for 
conversations about quality enhancement. 


Student engagement is a complex phenomenon. Institutions will have access to 
other data that could support their inquiry and strategy selection — the data of their 
benchmarking partners, retention data, student evaluations of teaching, perhaps 
data from another survey or focus groups on orientation activities or a work- 
integrated learning programme. An institution’s student engagement data are a 
valuable resource that offers a particular and important perspective, but it will be 


one resource among many. 


Student engagement data are particularly helpful in identifying how different groups 
of students evaluate their experience. These differences allow users to identity 
institution-wide strategies that address shortfalls in student engagement for specific 
groups, such as first-year students or international students. For example, if only 25% 
of an institution’s first-year students report using student learning support services 
often or very often, a general strategy may be required to promote greater use 
of those services. A closer look at survey results may reveal that 40% of first-year 
international students make use of these services, but only 20% of domestic students 
do so. This might suggest that targeted strategies are required. 


Institution-wide strategies can substitute for localised strategies when they are 
needed. Suppose 25% of an institution’s first-year students in humanities fields report 
working with other students on projects during class. Yet an institution-wide analysis 
notes that 40 to 60% of students in other fields of study report working with other 
students on projects during class. These data might suggest that a specific focus is 
needed in some faculties or departments to consider how tutorials are delivered or 
how assessments are designed. 


While a localised response may be required in the example mentioned above, local 
responses have a much better chance of success if there is a supportive institution-wide 
approach to student engagement. It is likely that an institution-wide approach will be 
needed for the analysis and reporting of survey outcomes, to support monitoring and 
evaluation of changed practice, and to explain the evidence base and the principles 
of student engagement to specific groups of academic and professional staff. An 
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institution may undertake an institution-wide audit of existing practices against key 
principles of student engagement. The institution could then establish specific and 
regular reporting mechanisms, such as annual student engagement reports. Student 
engagement could become a standing item on appropriate academic, management 
and business committees. Other options, which have both symbolic and practical 


impacts, include establishing: 


= a university office for student engagement; 
= institutional awards for student engagement; and 


= a fund for developing student engagement practices, or for pursuing research on 
aspects of student engagement. 


Benchmarking for continuous improvement 


Benchmarking can deepen the analysis and interpretation of data on student 
engagement, and what it can yield. Student engagement data can be benchmarked 
using criterion- or norm-referenced approaches. Benchmarking might take two 
hours or a year, or ten — each institution decides what is appropriate. Benchmarking 
activities offer opportunities to involve students in the collection and analysis of data. 


Criterion-referenced benchmarking involves establishing a target, usually a specified 
minimum outcome, or an outcome an institution aspires to. For example, the student- 
staff interaction items in the NSSE ask students to assess the frequency and nature of 
their contact with teaching staff. Data for an institution or faculty may show that 25% 
of first-year students report they “Often” or “Very often” have contact with teaching 
staff outside of class. An institution might decide, perhaps as part of a first-year 
strategy, that 50% is a preferred result on a metric of student and staff interaction. 
By nominating the preferred outcome — in this case, 50% — an institution establishes 
a criterion, a benchmark they aspire to. Against the criterion they can measure the 
effectiveness of changes put in place to increase the proportion of students who 
report that such interactions occur “Often” or “Very often”. 


Norm-referenced benchmarking involves comparing the institution’s outcomes with 
other universities or faculty outcomes with kindred faculties in other universities. 
Normative benchmarking can be undertaken simply by comparing the institution’s 
outcomes with the national or international results. This can give a general idea of 
the institution’s relative performance on specitic scales and items. 


A deeper sense of the university’s or academic staff’s performance can be gained 
through active benchmarking of their arrangements with selected universities or 


faculties. For example, an academic or professional staff member in a law faculty 
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is likely to significantly increase the power of data analysis and interpretation by 
comparing the law faculty’s outcomes with those of a law faculty in one or more 
other universities. This kind of normative benchmarking would extend good practices 
of student engagement in legal disciplines. It can also help to determine appropriate 
targets for criterion-referenced benchmarking in the faculty. 


Benchmarking relies on the willingness of benchmarking partners to share data. As 
the data belong to an institution, the institution then decides which results to share. 
Initially, an institution might agree to share only scale results rather than results for 
each survey item. They might decide to share only results for certain scales, however, 
as their confidence grows, they may agree to share more. 


It is especially helpful to compare results over time. To this end, benchmarking 
partnerships are usually long-term arrangements. Benchmarking partners usually 
establish a benchmarking agreement, covering matters such as: 


= confidentiality of an institution's shared data; 
"confidentiality of the discussions the institutions have about the data; 
a how benchmarking with partners will be conducted; and 


= the schedule for benchmarking activities. 


Every faculty in every university — such as business, law, science, medicine — is different 
in important ways. Benchmarking allows them to tease out the influence of those 
differences on their outcomes. If an institution’s business faculty operates across two 
campuses and one of their benchmarking partners offers business programmes on 
one campus, they can explore through discussion what effects these arrangements 
might have on their respective outcomes. 


Students in different disciplines manifest different patterns of engagement. If results 
indicate that those studying information technology are frequently less engaged 
than those studying natural and physical sciences it would thus make sense for 
an information technology department to seek another information technology 
department as a benchmarking partner. Through these discussions they can 
better identify influences on student engagement for students in their discipline. 
An institution can assess impacts for information technology students of varied 
approaches to work-integrated learning, for example, or the creation of supportive 


learning environments. 


Such discussions can reveal ideas about, and perspectives on, student engagement 
that are new and helpful. Benchmarking can offer ideas for adjusting an institution’s 
practice based on the practical experience of their partners. Partners often develop 
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better approaches through discussion and sharing of practices. Benchmarking is a 
clear indication to students, to accreditation and quality assurance bodies, and to 
the institution’s professional networks, that the institution takes student engagement 
seriously. An institution can report outcomes of benchmarking discussions to these 
groups, if the benchmarking agreement covers this. 


Selecting appropriate benchmarking partners is an important decision. It can help 
if benchmarking partners are similar in some ways. An institution might consider 
faculties or departments with similar characteristics such as student numbers, student 
demographics (for example, proportion of students who are first in family to attend 
university, or proportion of students studying full-time), or location (for example, 
provincial or inner-city). A faculty might consider faculties that are strong pertormers 
on scales or items where the faculty’s performance is not as strong as they would like. 


Process benchmarking is another option. Sometimes an institution may wish to 
explore differences or similarities in survey outcomes at greater depth. Perhaps 22% 
of an institution’s first-year education students respond “Quite a bit” or “Very much” 
in response to an item about whether an institution emphasises providing the support 
students need to succeed academically. An institution’s benchmarking partners may 
have outcomes ranging from 42% to 56% on this item. Yet, based on discussions, 
the institution believes that the range of support services they offer is as wide and 
deep as their partners. An institution could seek the agreement of their partners to 
study how they link their first-year students with these services, how those services are 
managed, and how they are delivered. 


In effect, an institution can process benchmark their own services against their 
partners’ services to determine if there are improvements they could make. 
Benchmarking can also reveal fertile areas for research activity, which contribute to 
advances in the scholarship of teaching and learning. This research could be within 
a school or department, or cross-institutional within a discipline. It may be research 
that seeks to describe or explain exceptional and consistent outcomes, or it may be 
research that seeks to describe the impact of changes to practice that an institution 
introduces. 


Broadening staff involvement in student learning 


Student engagement data can be a key input to reviewing entire courses of study, 
revising assessment designs for first-year students, or revising feedback practice in 
a single subject. The evidence base that underpins the NSSE and its adaptations 
illustrates a clear link between student engagement and learning experiences that 
are challenging, enriching and supportive (McCormick, Kinzie & Gonyea 2013). 
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Exploring an institution’s data with these aspects (support, challenge and enrichment) 
in mind can suggest areas for extension or improvement. Radloff and Coates (201 4) 
indicate, that there is a strong correlation between satisfaction and positive responses 
on engagement questions about applying theories or concepts to practical problems 
or in new situations. Improving the proportion of positive responses to such item 
could involve a minor or a major review of pedagogical approaches and materials, 
such as case studies and tutorial problems. An institution can explore options that go 
beyond the classroom in pursuit of enhancing student engagement. For example, staff 
could design assessment tasks that can be completed through voluntary activities with 
community organisations, or in work placements. This kind of change is valuable. 
The challenge is to broaden teaching involvement with students beyond the familiar 
boundaries of coursework and classroom — to see learning, and an academic’s 
role in supporting learning, in a wider context. An example of this is an item that 
taps into whether students received prompt written or oral feedback from teachers 
on academic performance. Feedback practice need not be limited to conventional 
activities like oral feedback on a group tutorial presentation, or written feedback 
on a test or assignment. A department or school could, for example, institute a 
debating event in a subject twice a semester with two teams of three tackling an 
impromptu topic. Each team might comprise two students and one academic from a 
department or school. A programme leader could be the moderator, or perhaps the 
head of school or head of department could fill this role, with moderation involving 
commentary on the debate and a short email to the student team members assessing 
the content of their presentations. The emphasis is on enjoyment, the outcome is 


learning, and the feedback is valuable. 


A voluntary seminar programme for first-year students is another forum in which 
students could receive valuable feedback that is not linked to graded assessment. 
Students are valuable contributors in such activities, and academics can create 
challenging, enriching and supportive learning experiences by seeing students as 
competent contributors to the work of the university. Peer tutoring is a great example 
of this but it may not be an option that appeals to all students, or that all students 


are able to participate in. 


If an academic sees students as formative contributors, and accepts that students are 
interested in challenge and enrichment, there are many options that can broaden 
an academic’s involvement in student learning beyond coursework. For example, 


students could: 


= take guided responsibility for literature searches and literature reviews; 


140 


LEVERAGING STUDENT ENGAGEMENT EVIDENCE FOR INSTITUTIONAL CHANGE AND IMPROVEMENT 


= assist in editorial activities related to academic journals, such as layout, checking 
of reference lists to ensure they conform with the relevant style, and cross-checking 
in-text references with reference lists; 

= assist with translation of documents related to academic research; 

= introduce speakers at public lectures, and offer acknowledgements; and 

= play a substantial role in promoting the study of science and mathematics in primary 
and secondary schools as part of the university’s community engagement activities. 


Such contributions require preparation and guidance from academic and professional 
staff. Developing such contributions as ongoing activities in a subject, department or 
faculty will lead to a growing fund of knowledge about how to structure preparation 
and guidance efficiently. Students who contribute in such ways can become part of 
the training effort for students who follow them. Inevitably there will be successes and 


some stumbles. 


Lecturers can make a difference well beyond the classroom, and the literature on 
student and staff interactions that underpins contemporary student engagement 
research indicates that this is where many of their most formative contributions can be 
made. Innovation in teaching practice that promotes student engagement does not 
need to be confined within the boundaries of coursework. Innovation beyond those 
boundaries is likely to improve student engagement and, consequently, contribute to 


improved learning outcomes. 


Enhancing interactions between students and staff 


Much work on student engagement emphasises the importance of engaging 
students outside classes. Many academic and professional staff are generous with 
the time they commit to interacting with students. It may be found, for example, that 
21% of students had worked with teaching staff on activities other than coursework, 
and that 44% of students reported talking about career plans with teaching staff 
or advisors. The complement of these figures, of course, is that 79% of students 
had not worked with staff on activities other than coursework, and 56% of students 
had not discussed career plans with teaching staff or advisors. Yet broad research 
evidence suggests that high levels of student-staff interactions have positive effects 


on learning, motivation, and persistence — on engagement. 


Results from student surveys may be used to target wider engagement. When 
considering how to promote greater staff-student interaction, it is important to go 
beyond preconceptions that limit thinking and action. A common preconception is 
that many, or even most, students are rarely on campus. The presumed corollary is 
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that because students are absent they have no time to interact with staff outside of 
class time. Survey data can provide a check on the veracity of opinions like these. 
Survey results may indicate that 22% of domestic students and 36% of international 
students spent more than ten hours a week on campus outside class time. 


The student-staff interaction scales used in the various national implementations 
comprise around six items. An institution’s survey report provides data on the scale 
of each constituent item, and with a breakdown of the data for first-year students 
and later-year students. Data are also reported for other variables like field of study 
and study mode (part-time, full-time or distance). An institution can use these data to 
establish, for example, how many hours outside class time their later-year international 
students studying information technology spend on campus. It can help to compare 
an institution’s data with national outcomes and with benchmarking partners. The 
comparison might lead an institution to set a formal target for improving the outcome 
on the student-staff interaction scale. Equally, an institution can approach the task 
incrementally by trying a range of interventions on a trial basis and examining what 
strategies make a difference, and then expanding the successful ones. 


Student engagement data may offer an effective way of reviewing existing strategies, 
however, very often the challenge is to make these existing engagement strategies 
work more effectively. It may be that 30% of an institution’s first-year students in 
management and commerce answered either “Often” or “Very often” to an item 
about discussing grades or assignments with teaching staff. The institution needs to 
decide if that percentage is a satisfactory outcome. The institution may investigate 
further and find that although all lecturers and tutors have scheduled hours for 
student consultations, very few students make use of the opportunity. They may 
find that first-year students feel uncomfortable about taking up the opportunity 
— students may not know anyone else who has done so, they may be uncertain 
what happens during a consultation, or they may be unsure about the benefits of a 
consultation. Academics and educational support staff can then begin to think about 
specific actions that might encourage students to take advantage of the opportunity 
by lessening anxiety. A description provided in a lecture or tutorial perhaps, or a 
testimonial report from students who have scheduled a consultation, or a video clip 
of a consultation accessible on the university’s learning management system are 


examples of possible actions. 


Student-staff interactions are often thought of in formal terms, such as ‘scheduled 
consultations’. Such opportunities are evidently fundamental to education, but it is 
also important to develop informal engagement strategies. Further, it is useful to 
consider how informal interactions can become a larger part of the student experience. 
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Opportunities for informal interactions open possibilities for conversation across 
a wide range of topics. These interactions could introduce students to previously 
unexplored ways of engaging with their subjects and disciplines, and place the 
learning project in a relaxed context which supports engagement. 


Informal opportunities could be as straightforward as having coffee with a small 
group of students now and then, and guiding conversation towards the career 
options a discipline leads to. Another option may be inviting three or four students to 
a lunch with a visiting scholar who has expertise in the topic for their team project. 
Another example may be running a late afternoon review session in the week before 
the first-year exam and ending the session with delivered pizzas. These are but a 
few possibilities; specific opportunities are best developed with sensitivity to student 
needs and institutional culture, and in consultation with student representatives. 


It is common to think of staff/student interactions that promote student engagement 
as taking place only between academic staff and students. Professional staff can also 
play a role well beyond offering a scheduled learning support activity, or dealing 
with students on a formal, transactional and administrative basis about matters 
such as enrolment, special consideration or course advice. Finding ways to include 
professional staff in student-staff interactions can produce a sense of ‘us’ rather than 
‘me and them’, and also reinforce shared responsibility for student success across 
functional areas and job descriptions. 


Monitoring quality data over time 

As with any survey, the impact is most powerful when institutions examine how results 
change over time. Monitoring results over time allows institutions to: 

= identify areas for further growth; 

= recognise and celebrate achievements; 

= identify areas where performance seems to be declining; 

= monitor progress toward improvement targets; 

= identify professional development priorities for academic and general staff; and 


= demonstrate quality improvement to students, to accreditation and quality assurance 
bodies, and to professional networks. 


For example, monitoring the Work Integrated Learning scale, which forms part 
of the Australasian Survey of Student Engagement (AUSSE) over five years would 
highlight how survey results become more powerful over multiple administrations 
(ACER n.d.). It might be that the student perceptions of the work integrated learning 


programme over two years are not improving, but moving within a consistent range. 
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It might be that in a specific year the institution introduced an improved approach to 
work integrated learning which accounts for an observed improvement. Subsequent 
years’ results may be better still, perhaps because the institution ironed out some 
teething problems encountered. Only people on the ground know the story behind 
the numbers. Only they can tell it. Every university, every faculty, is different. None 
of this means that the first two years’ results are useless. The first two or three 
years of any data collection need to be seen in the context of the long-term quality 


improvement cycle. 


CONCLUSION 


As suggested in this chapter, rhetoric about student engagement can often be removed 
from the realities of institutional contexts, open to multiple perspectives and used for 
a variety of disparate purposes. For these reasons, it can be a challenging to attempt 
to anchor student engagement within the individual environment of each institution. It 
is helpful to identify clear strategies for improving student engagement. This chapter 
showcases a small number of ways that people with responsibility for leading and 
managing student engagement can structure their work. We reviewed developing 
role-based and institution-wide approaches to improving student engagement. Such 
improvement work does not require substantial or complex interpretation to guide 
change. Student engagement data are designed to be practically focused, typically 
linking directly with specific individual or institutional conditions or practices. In 
principle, therefore, it becomes easier for a broad range of stakeholders to review 
survey results and identify contextually relevant responses. 


Unfortunately, evidence-driven change is usually difficult to achieve. Data from 
feedback surveys are only a small facet of broader institutional contexts in which 
many complex factors are at play, and such data can itself be indeterministic. Data 
pointing to low levels of interaction between students and staff may imply staff 
disengagement or student disengagement or the presence of practical obstacles 
that neither party can work around, or perhaps even a nuanced situation that 
goes undetected by a few individual survey questions. Understanding the effective 
transference of data into effective change is a significant research activity, and one, 
which this summary chapter has only begun to touch. 


The suggestions are not intended to be exhaustive or essentialist. Rather, they 
are designed to prompt imagination in ways that spur innovative approaches to 
monitoring and improving engagement. Ultimately, the approaches discussed 
here seek to energise one broad agenda with the aim of enhancing engagement — 
encouraging people who have the potential to make a difference to take evidence 
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on student engagement seriously. The quality improvement literature suggests that 
there are an infinite number of ways this might be done, and in closing the current 
analysis we encourage creative consideration of further effective approaches. Are 
there particular approaches to recurrent or performance funding, for instance, 
which are likely to enhance engagement? What generalisable reporting designs 
and approaches are likely to spur the right kinds of action? What would student 
engagement frameworks that map out policies and practices in ways that signpost 
and prompt increasing levels of growth look like? What forms of cross-professional 
collaboration are likely to stimulate engagement? 


It is important to recall that student engagement is an inherently contextualised 
phenomenon that requires regular reconceptualization, redevelopment, and 
redeployment. The advent of large-scale relatively open online education, for 
instance, creates new forms of engagement, unturls new risks, and necessitates new 
forms of support. Engagement research has yet to yield a single ‘silver bullet’ that 


solves the engagement riddle. 
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THE ROLE OF STUDENT AFFAIRS 
IN STUDENT ENGAGEMENT 
AND LEARNING 


Vasti Torres & Matete Madiba 


Aman cannot sit down alone to plan for prosperity — Nigerian proverb 


INTRODUCTION 


For many countries, the belief that the United States has a respected and recognised 
student affairs profession makes the literature in that country the main source for 
guidance within higher education (Dungy & Gordon 2011; Schreiber 2014a). 
Student affairs in the United States does have a longer history than in South Africa 
and other countries, yet the role it plays in higher education has several similarities. 
This chapter seeks to consider the role engagement data can play in assisting 
student affairs to advance student learning. This, we argue, should be one of the 
most important focus areas for student affairs practitioners in both the United States 
and South Africa. To accomplish this, the chapter first considers the philosophical 
origins of student affairs in both the United States and South Africa. This is followed 
by a discussion on the evolution of student learning as a central tenet in the work of 
student affairs practitioners. The chapter concludes with an explicit connection with 
the data provided in student engagement surveys and the role student learning has 
within student affairs programmes. 


STUDENT AFFAIRS’ INFLUENCE ON LEARNING WITHIN THE UNITED STATES 
CONTEXT 

The Student Personnel Point of View published by the American Council on 
Education (ACE) articulated the role of student affairs within the United States as 
focused on the “whole student” and the services that personnel administrators 
provided (ACE, 1949). The notion of how to care for the whole student has evolved 
as a result of experience and student demographic shifts that occurred within higher 
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education over time. As these philosophical ideals evolved, the focus on student 
development theories allowed student affairs practitioners to understand how 
students develop (or mature) and how their development influences their ability to 
learn within the higher education environment. It was the 1994 publication, The 
Student Learning Imperative: Implications for student affairs, that first explicitly 
acknowledged that “the concepts of ‘learning’, ‘personal development’, and ‘student 
development’ are inextricably intertwined and inseparable” (American College 
Personnel Association [ACPA] 1994:3). This philosophical document highlighted the 
synergistic relation between what students learn in the classroom and what they learn 
outside the classroom; therefore, acknowledging that learning takes place as a result 
of interactions between students and the multiple environments in which they exist, 
including individuals such as academic and support staff, and peers. The clarion call 
that this document put forth was that student affairs has the responsibility to design 
and manage educationally purposeful activities that promote student learning. 


In 1998 the joint report Powerful Partnerships: A Shared Responsibility for Learning 
(AAHE, ACPA & NASPA 1998) followed up with the ideals that collaboration between 
student affairs and academic staff would further our ‘collective wisdom’ to meet the 
challenges that our students will face in a diverse world. Later, the book Learning 
Reconsidered: A Campus-wide Focus on the Student Experience (Keeling 2004) 
further echoed that calling for educationally purposeful activities within student 
affairs. Understanding that student learning is the primary focus of all divisions 
within a higher education institution is only the first step in enacting this vision. For 
many years, higher education, as an enterprise, understood that learning was the 
desired outcome of formal education, but there was less focus on the process of 
how this learning took place. In the United States, multiple groups from accrediting 
bodies to individual institutions, began requiring that clear learning objectives for 
students be explicitly communicated through syllabi and programme objectives. This 
required that both academic and support staff within higher education understand 
the process of how students learn and how we know they are learning the material 
to promote the desired outcomes. 


Despite the information available to student affairs practitioners in the United States 
about how to influence student learning, there continue to be barriers to fulfilling 
the vision of student success through intentional educational programmes. There 
are four common ways that practitioners may unknowingly present barriers for 
student engagement: 


= Rather than holding students to high expectations to promote student success, 
student affairs practitioners may overemphasise the support element and not 
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provide the level of challenge (dissonance) needed to encourage critical thinking 
and development (Whitt 2005). 

= At many institutions, the types of programmes emphasised in orientation and other 
departments continued to be more focused on students’ social lives rather than on 
educationally purposeful activities. 

= When requesting additional funding for student affairs programmes, the language 
used to justify the funds continues to be primarily focused on support rather than on 
assessment of how programmes promote student learning through engagement. 

= By constantly comparing the student affairs division to the academic affairs functions, 
practitioners miss the opportunity to articulate their own unique contributions to the 
student learning process. 


While other countries may see student affairs in the United States as a well-established 
and recognised profession, they may mistake the level of professional training 
required as providing equal respect with academic affairs within institutions of higher 
education. The reality is that despite the advances in theoretical foundations (e.g. 
Schuh, Jones & Torres 2016), professional competencies (ACPA & NASPA 2015), 
and programme standards (Council for the Advancement of Standards in Higher 
Education [CAS] 2015), student affairs continues to take a back seat within the 
primary academic mission of many higher education institutions. To better integrate 
student affairs into the academic mission requires that the work of practitioners be 
framed differently to focus on student learning as the primary role for all out of class 
programmes (ACPA 1994). Regardless of the historical background and professional 
training available to student affairs practitioners in the United States, there continues 
to be a need to explain the learning that occurs out of the classroom. Whether it 
is in the United States or another country, those of us who believe in the powerful 
nature of the learning environment outside the classroom must continue to work on 


explaining how we contribute to student learning through engagement. 


STUDENT AFFAIRS IN THE SOUTH AFRICAN CONTEXT 


Mandew argues that “student affairs in South Africa is arguably as old as the university 
and has largely played a supportive role that has not always demanded critical, 
rigorous and intellectual reflection on practice and mission” (Mandew 2014:56). 
The critique offered by Mandew alludes to the less visible and less impactful role that 
student affairs has played in South Africa. 


Unlike a strong movement towards framing student affairs around student learning, 
such as has been happening in many parts of the United States, student affairs in 
South Africa has not made the shift in focus from basic support to student learning. 
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In fact, the development of student affairs as a key contributor to student success, 
development and support has only recently been recognised. As the most direct 
link between student leaders and institutions, student affairs units have had to play 
a strong political role as mediators during South Africa’s pre-democratic apartheid 
era. This role intensified during the student anti-apartheid uprisings during the 
1980s as pressure to control students increased (Bodibe 2009; Schreiber 2014b). 
The mediating role between students and institutions these units have come to be 
known by, as well as the diversity of professional qualifications making up the staff 
component of student affairs fuel long-standing stereotypes such as “being the fire 
brigade that is called upon to extinguish or at least dampen student protest, with too 
little appreciation of the overall critical work being done consistently year in and year 
out” (MacMaster 2014:30). Moreover, Speckman and Mandew (2014. P x) bemoan 
the “sad realization that ... there has been little to no student affairs literature coming 
from South Africa”. Schreiber reveals that establishing student affairs as “a discipline 
and profession” is among the challenges faced by student affairs in developing 
countries. She further argues that these countries need to “engage in local theory 
and critically engage with established theoretical frameworks”, including the need to 
articulate a coherent and “normative meta-framework that accommodates multiple 
indigenous realities which need to flourish in a global context” (Schreiber 2014a:21). 


These factors contributed to a sense of a ‘lack of identity’ of student affairs for many 
years — up until a strong national focus from the early 2000s on widening access 
and promoting student success, which placed student affairs divisions in key roles for 
support and development. This also prompted a movement to professionalise student 
affairs in South Africa. The Journal of Student Affairs in Africa (JSAA), launched in 
2013, provided a much-needed platform for student affairs scholarship to develop, 
and postgraduate qualifications aimed at student development were introduced at 
two universities. In 2012, the Southern African Federation for Student Affairs and 
Services (SAFSAS) was established to collaborate on strategies to enhance student 
development and support. SAFSAS serves as an umbrella body to the following 
professional bodies: 


® South African Association of Senior Student Affairs Professionals (SAASSAP); 


* Southern African Association for Counselling and Development in Higher Education 
(SAACDHE); 


= National Association of Student Development Practitioners (NASDEV); 

= South African Association of Campus Health Services (SAACHS); 

= Association of College and University Housing Officers — International (ACUHO-I); 
= Higher and Further Education Disability Services Association (HEDSA); and 
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= Financial Aid Practitioners of South Africa (FAPSA). 


The SAFSAS mandate reads: 


As professionals responsible for the administration and provision of 
Student Affairs and Services, focussing primarily on co-curricular activities 
within institutions of higher learning, we are committed to identify with 
practices that will move our professions forward. We believe in serving 
students holistically, so that they become well-rounded and thoroughly 
grounded graduates and responsible citizens. We are committed to the 
principles of good governance, service delivery and leadership in student 
affairs and services, including: 

a Engaging students in active learning. 

= Helping students to develop coherent values and ethical standards. 

= Setting and communicating high expectations for student learning. 

= Using systematic inquiry to improve student and institutional performance. 
= Using resources effectively to achieve institutional missions and goals. 
= Forging educational partnerships that advance student learning. 


= Building supportive and inclusive communities (SAFSAS n.d.). 


The mandate makes it clear that SAFSAS aims to steer student affairs practices 
towards facilitating student learning, while also contributing to developing national 
and institutional graduate attributes and providing a ‘safe space’ for students to feel 
supported. The strong inclusion of student learning as a priority for student affairs 
has up to now been neglected for the historic reasons mentioned earlier, but also 
because of the parallel development of academic development (AD) within different 
institutional support divisions or faculties. 


There are several observations one can make about student learning as a focus in 
higher education. ‘Student learning’, when defined in a narrow way, is confined to 
the formal classroom and allocated to the territory of teaching staff (academics). All 
the other role players within higher education are seen as being on the periphery of 
student learning, or irrelevant. This is well illustrated by the history of AD. Lorraine 
Stefani describes the challenge academic development has faced over the more than 
AO years of its existence. She articulates the gains and the advances made within 
the area, including the fact that AD “activities and centres are now commonplace 
in most countries”, and she puts South Africa in that list (of countries). She further 
indicates that AD has “moved from being a highly marginal practice to a dominant 
discourse framing the ways university teaching is understood” (Stefani 201 1:3). 
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This positioning of moving to a dominant discourse has not come easy even though the 
impact is visible. According to the Council on Higher Education (CHE), in South Africa, 
one of the biggest achievements is how AD has led to policy initiatives around the 
funding of foundation programmes and influenced the need for national curriculum 
reform considerations (CHE 2013). Even with these achievements, AD and its 
practitioners have not won the struggle of being marginalised completely; it is a matter 
of ‘aluta continua’ (‘the struggle continues’). Stefani continues to indicate that one 
area that is still problematic for AD is the lack of coherence in the purpose of AD and 
the issues “are compounded by the lack of an agreed framework for evaluation of the 
impact, value and added effectiveness of academic development” (Stefani 2011:4). 


As Mandew (2014) asserts, AD, as a movement in South Africa, offers several lessons 
to student affairs. The challenges are becoming clearer and student affairs staff 
need better framing and recognised ways of assessing and evaluating impact, value 
and effectiveness. In exploring evaluation and assessment approaches with the AD 
movement, Stefani identifies at least two paradigms that are in operation: one focused 
on “continuous improvement of student learning outcomes and student experience”, 


and the other that focuses on the accountability “regime” (Stefani 2011:12). 


Student affairs in the South African context is at a far more challenging position than 
academic development when it comes to student learning. The narrow definition of 
student learning referred to earlier creates some distance between student learning 
and student affairs. This is because student affairs is seen as focusing on out of 
class activities, more specifically in the sense of extra-curricular activities, and only 
grounded in providing support to students. Because AD is an established field in 
South Africa, it becomes an extra factor that adds to the distance between student 
affairs and student learning (in its narrow sense). Numerous activities that student 
affairs practitioners could be doing or overseeing in relation to student learning are 
allocated to or taken on by AD. These include activities such as taking responsibility 
for mentoring and tutor programmes (recruitment, training and evaluation of tutors). 
These are the responsibility of AD centres and divisions. Other examples are first- 
year orientation programmes and the first-year experience (FYE) suite of offerings. It 
is striking that in in most South African universities where such practices exist, first- 
year orientation is (still) with student affairs, while the FYE (and its offerings) is with 
AD centres. 


This is in stark contrast with the United States context, which does not have organised 
AD in the way that Mandew (2014) or Stefani (2011) describe and characterise. 
Student affairs in the United States is at a far better position to lessen the distance 
that might exist between student affairs and student learning. One might argue that 
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in the South African context, student affairs must compete in some instances, or 
collaborate in some way with AD in order to be closer to student learning in terms of 
being trusted with relevant responsibilities and operations. On top of having to deal 
with the same challenges as faced by AD, student affairs must compete with AD, but 
from a weaker position given that it (student affairs) has not yet established itself as 
a profession and lacks its own theoretical and evaluation frameworks. Whereas it 
is relevant in the United States context to make the ‘clarion call’ that student affairs 
has the responsibility to design and manage educationally purposeful activities that 
promote student learning, it is somehow complicated to make the same call within 
the South African context. 


Another important factor guiding the focus of student affairs in South Africa is the 
state of vulnerability of students entering higher education. This in terms of socio- 
economic circumstances and/or academic underpreparedness (because of poor 
quality schooling, or having to adapt to social and cultural circumstances very 
different trom home). A national study on quality enhancement of public universities 
asked institutions to list the main dimensions impacting students’ lives and their 
ability to succeed at university. The dimensions listed (CHE 2015:22) are: 


— 


. understanding of subject matter; 


2. competence in various academic literacies, including academic 
reading and writing, quantitative literacy, information literacy and 
computer literacy; 


use of effective study skills; 

sense of vocation and identification with the chosen career; 
physical health; 

psycho-social-emotional well-being; 


facilitation in several life skills; and 


Oey Oy Or 


provision of material needs, including funding, accommodation, 
food and transport. 


These dimensions point directly to the socio-economic pressures students face, as 
well as the large transitional gap between poor-quality schooling and university 
studies. The significant amount of support most students in South Africa require 
guides the development of student affairs towards an equal prioritisation of both 
student support as well as facilitating learning and development. Moreover, a 
major challenge for institutions, and particularly student affairs, is the persistence of 
structural inequity. Students are often blamed for disadvantages beyond their control, 
such as receiving poor-quality schooling, which impacts on the selection of subjects 
as well as their performance in these subjects when entering higher education. These 
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disadvantages are often perpetuated through insensitive institutional cultures and a 
lack of understanding around the academic and non-academic challenges these 


students face daily (Madiba 2014). 


From the well-established knowledge base the United States created over the 
years, they appear to have much to offer in terms of student affairs as a profession 
because of its long history and rich literature. An organisation like the National 
Association of Student Affairs Personnel Administrators (NASPA) founded in 1919 
boasts of 15 000 members and 2 100 institutions across 50 states and 25 countries 
(NASPA, n.d.). However, as pointed out earlier in this chapter, even with such 
elaborate developments, student affairs in the United States continues to take a back 
seat within the primary academic mission of many higher education institutions. It is 
evident that the role of student affairs here in South Africa and abroad needs framing 
and a better focus. One such way is to build from what we can learn from large data 
sets that are available from relevant research projects. 


USING STUDENT ENGAGEMENT DATA TO INFORM STUDENT AFFAIRS PRACTICE 


Research has illustrated that the time and energy students devote to educationally 
purposeful activities is the single best predictor of their learning and personal 
development (Kuh, Kinzie, Schuh & Whitt 2010). By considering student engagement, 
institutions can begin to gauge the time and effort students put into their academic 
work, as well as outside of the classroom activities, and determine if institutional 
resources and organisational programmes induce the level of engagement needed 
to promote student success (Kuh et al 2010). McCormick (2009) as well as Strydom, 
Kuh and Mentz (2010) argue that even though student engagement research is 
often used internationally for external accountability purposes, the real value 
of engagement data lies in how it contributes to internal self-reflection of those 
responsible for undergraduate education and inter-institutional benchmarking. 
Strydom and Mentz (2013) interpret this as contributing to a culture of evidence, 
which helps institutions evaluate their own strengths and weaknesses regarding 
student success. 


Data from the South African Survey of Student Engagement (SASSE) could be used 
to facilitate such reflection within South African student affairs divisions. In using the 
results, these divisions can better understand to what extent their intentions to assist 
students to persist and succeed are realised. There are at least two mechanisms that 
can help to analyse and review the data to help student affairs programmes. 


The main mechanism available to institutions that use the survey of student 
engagement is the ability to disaggregate the data in two critical ways. First, the 
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ability to look at the data by those who use or participate in the programmes 
allows practitioners to determine the value of their programmes through student 
engagement indicators. This is accomplished by using the student engagement 
indicators as the outcome variable and comparing those that use or participate 
in programmes with those that do not. If participating in a programme or using a 
particular service does not seem to influence the appropriate engagement indicators 
in a positive manner, then adjustments and improvements should be made to that 
programme or service based on further enquiry as to why the programme did not 
contribute to engagement. 


The second data elements that should be carefully analysed with the student 
engagement results are to disaggregate by race, gender, and other relevant factors 
impacting on students’ experiences in higher education. Knowing how different types 
of students respond to the programmes and services is critical to serving all students 
within our institutions. Analyses in the United States illustrate that there are high- 
impact practices (HIPs) that students can engage in that further student success. These 
practices tend to have greater self-reported gains for underrepresented students 
within higher education (Finley & McNair 2013). HIPs work to engage students 
because they tend to promote high levels of student-staff interaction, reflection on the 
learning process, and provide both challenge and support for the learning process. 
These types of activities can promote greater success for underrepresented students 
because they allow first-generation university students to have greater interaction 
and support to learn how to be good students. Consistent with Kuh’s (2011) ideals 
about teaching students how to use resources, many students who do not have 
the social knowledge about resources can learn through educationally purposeful 
activities that promote interactions with academic and support staff, which in turn 
could impact their own learning and success. Madiba (2014) rightfully suggests 
that students are blamed for disadvantages beyond their control. HIPs can provide 
a strategy for institutions to help disadvantaged students gain the social capital 
needed to succeed in postsecondary education. 


Within the United States, the HIPs that work for improvement of student success 
include first-year seminars, writing-intensive courses, collaborative learning 
assignments, internships, service learning, undergraduate research with a lecturer, 
study abroad, learning communities, and capstone courses or projects (Kuh, 
O'Donnell & Reed 2013). Chapter 5 in this book expands more on developing 
HIPs in the South African context. The inclusion of 11 identified practices in the 
2015 SASSE administration provides a conversation starting point about developing 
and implementing contextualised HIPs, however, linking these practices to student 
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affairs is too early. The data presented in this chapter, therefore, focuses on how 
student engagement data can help student affairs divisions gauge where they need 
to intervene in order to help students develop and learn. 


WHAT DOES STUDENT ENGAGEMENT DATA TELL US ABOUT STUDENTS? 


The measurement of student engagement in South Africa is a far newer practice than 
in the United States. The measurement was piloted in 2007 at seven South African 
institutions and has been subject to several processes of contextualisation. For the 
purposes of this chapter, data from the 2014 administration of the SASSE are used. 
The sample consists of 2 935 first-year students and 9 291 senior students. Males 
represented 47% of the sample and females 53%. Two-thirds of the sample consisted 
of black Africans, followed by white (14%), and 14% making up the other races. 
First-generation students represented just over 70% of the sample. 


To illustrate how student engagement data could contribute to student affairs’ 
knowledge about student learning and development, we draw from SASSE items 
enquiring about the extent that institutions emphasise certain behaviours as well as 
the quality of their interactions with support divisions. These include: 
# |ndicate the quality of your interactions with peer learning support (tutors, mentors, 

facilitators), and student support services (e.g. counselling, health, disability, career). 
a How much does your institution emphasise the following? 

— Spending significant amounts of time studying and on academic work. 

— Providing support to help students succeed academically. 


— Providing support for your overall well-being (recreation, health care, 
counselling, etc.). 


— Attending events that address important economic, political, or societal issues. 


The items were analysed to determine differences in scores reported for subgroups 
of gender, race, whether they live on or off campus, and generation status through 
independent t-tests and one-way analyses of variance where appropriate. Each 
analysis reports on the significance of difference between group responses as well 
as an indication of the practical significance through a calculation of the effect size 
(Cohen 1988). Figures 7.1 to 7.3 display the group means (out of a maximum of 4). 
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FIGURE 7.1 Quality of interactions with peer learning support and student support services 


In general, it seems as if students (N=12 226) feel their interaction with peer 
learning support (tutors, facilitators, mentors) and student support staff are of 
medium to relatively low quality (M=2.9) and (M=2.4) respectively. Independent 
t-tests show that there is no significant difference in how males and females rate 
the quality of experiences with peer learning support (o=.325) or student support 
services (p=.079). Statistically significant differences were found between the 
perceptions of quality interactions of students who live on campus and those who 
live off campus, with on-campus students rating the quality of interactions with peer 
learning support (op=.000; d=.08) as well as student support services (p=.000; 
d=.06) higher, although both findings reveal very small practical significance. First- 
generation students rate both these interactions of higher-quality (peer learning 
support: p=.000; d=.14; student support services: p=.000; d=.21), also with 


small to moderate practical significance. 


Students are asked to identify which racial group they identify with most, with options 
including “Asian”, “black African”, “coloured”, “Indian”, “mixed race”, “other”, 
“white”, and “l prefer not to answer.” For comparison purposes Figure 7.1 includes 
the groups indicating highest (black African) and lowest (coloured) mean scores. 
A one-way analysis of variance shows significant (p<.05) differences between all 
racial groups, with black African students reporting the highest quality of interactions 
with peer learning support (M=3), followed by Indian students (M=2.9), white 
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students (M=2.8), and coloured students (M=2.7). Asimilar pattern is seen regarding 
students’ perception of the quality of their interactions with support services, where 
black African students report a slightly above average score (M=2.5), while white and 
Indian students report a mean of 2.3, followed by coloured students (M=2.2). The 
practical significance of these differences between the group with the highest means 
(black African) and the lowest means (coloured) is moderate (d=.32) for both the 


quality of interaction with peer learning support, and student staff interaction (d=.27). 
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FIGURE 7.2 Institutional emphasis on studying and academic work, and providing 
academic support 


Figure 7.2 shows us that in general, students feel that their institutions place a 
lot of emphasis on studying and academic work, as well as providing academic 
support (M=3.4 and M=3 respectively). Significant differences, although of small 
to moderate practical significance, are found between on-campus and off-campus 
students’ indications of institutional emphasis on academic work (o=.000; d=.15) 
and support (o=.000; d=.19), with on-campus students indicating more emphasis 
on these factors from their respective institutions. Similarly, first-generation students 
significantly experienced more academic emphasis (o=.001; d=.02) and support 
(0 =.000; d=.17) than non-first-generation students; and women experienced more 
academic emphasis (o=.000; d=.14) and support (p=.000; d=.06) than males. 
These comparisons also revealed moderate to small practical significance in the 
differences between groups. Racially, black African students reported the highest 


mean for institutional emphasis on studying and academic work (M=3.4), which 
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differed significantly from all other racial groups, particularly the coloured group, 
(0 =.000; d=.28) with a moderate practical significance. 


Figure 7.3 shows that although a statistically significant difference was found between 
male and female perceptions of institutional emphasis on individual well-being as 
well as emphasis on attending important events, it proved to be of very little practical 
significance (o=.000; d=.08 respectively). 
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FIGURE 7.3 Institutional emphasis on providing support for overall well-being, and 
attending events that address important social, political or economic events 


First-generation students indicated significantly higher means related to how they 
perceive institutional emphasis on well-being and attending important social, 
political or economic events (0 =.000; d=.16 respectively), however, with a relatively 
low practical significance. On-campus students report higher means of institutional 
emphasis on their general well-being (p=.000; d=.27), with moderate practical 
significance. Similarly, on-campus students feel that their institutions emphasise 
attending social, political or economic events more than off-campus students 
(0 =.000; d=.28), also with moderate practical significance. Black African students, 
who indicated the highest mean score for institutional emphasis on general well- 
being, differed significantly from all other racial groups — and in particular the white 
group, who scored the lowest mean (o=.000; d=.52). This difference has a high 
practical significance. A similar finding is seen between the black African group 
scoring the highest mean for institutional emphasis on attending social, political or 
economic events, and the white group with the lowest mean (o=.000; d=.56). 
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DISCUSSION 


The examples of SASSE items explored here help us understand who our students 
are and how they are experiencing higher education. In general, the reported quality 
of interactions with student support staff was very low (M=2.4). While the general 
indication of quality interactions with peer learning support looks slightly better 
(M=2.9), there is still much room for improvement. While first-generation students 
show a greater indication of quality interactions with support staff, as well as feeling 
more supported academically, we need to ask whether we are doing enough to 
provide academic and non-academic guidance, support and opportunities for the 
development of this vulnerable group. On-campus students show higher quality 
relationships with peer learning support and student services staff, as well as feeling 
more institutional emphasis on academic support, general well-being, and even 
attending important events. In other words, the ease of reaching on-campus students 
perhaps promotes more engagement. One could assume that the ease of access 
and proximity to classes and support services could impact on these factors, namely, 
they spend more time on campus. However, data such as this pave the way for further 
inquiry into these differences. For example, if further studies find that on-campus 
students have a stronger sense of belonging, or make use of informal mentors in the 
residence structure, student affairs divisions could invest in innovative interventions 
targeting the development of these factors amongst off-campus students. Commuter 
programmes focused specifically on the off-campus population promote both a sense 
of community and engagement for these students. Finally, race continues to be a 
factor impacting students’ experiences. While it is good to see the positive responses 
from black African students in all these items, it is quite worrying to see how students 
identifying with the coloured group feel excluded, with low quality interactions with 
peer learners as well as support staff, and academic support. Similarly, and with high 
practical significance, white students feel that their institutions do not place much 
emphasis on their general well-being, nor do they encourage attendance of social, 


political, and economic events. 


It should be noted that there is inconsistency between the student ratings of institutional 
emphasis and students’ perception of the quality of interaction. While institutional 
emphasis was rated relatively high, the quality of interaction was not consistent. This 
should lead student affairs practitioners to carefully consider ‘how’ the programme 
is delivered, not just putting the programme in place. For interventions, including 
HIPs, to work effectively they must be done well. Student affairs programmes and 
services can provide quality assurance using SASSE data to ensure that programmes 


that engage students in learning activities are done well. 
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These analyses are just a few examples taken from support-based items of the 
SASSE to illustrate how data can help us to rethink interventions and where to 
focus our attention to help students succeed. Other items, such as those related to 
HIPs, are particularly important to gauge where co-curricular, scaled interventions 
could be implemented to promote student learning and development. Using data 
to document the influences that student affairs programmes and services have on 
student engagement is a strong first step in providing practitioners with the necessary 
language to articulate the purpose of student affairs. It is critical to understand 
that changing perceptions is more difficult than just providing data. To change the 
culture within our higher education institutions will require data and our ‘reculturing’ 
of student affairs divisions to understand that the student success mission is the 
most important goal we should all be working toward in our day-to-day activities. 
The available SASSE data offer useful benchmarks across a set of participating 
intuitions and these serve as national benchmarks. For student affairs specifically, 
the development and measurement of HIPs, as well as items related to students’ 
sense of support they receive from their institutions, and how they interact with this 
environment could provide meaningful, relevant and actionable data to guide co- 


curricular interventions. 


THE WAY FORWARD 


The urgency to help students succeed through higher education places significant 
pressure on student affairs in South Africa to hasten the project of establishing itself 
as a profession and finding theoretical frameworks that are critical and relevant to 
the local and global context within which it is located. Student affairs practitioners 
are often called upon to be “interpreters and advocates of student life and student 
issues”, which coincides with calls for student affairs to be more aligned with 
social justice goals, as well as being integrated into all facets of student learning 


(MacMaster 2014; Schreiber 201 4b). 


In the South African context, it is argued that the quality of ‘graduatedness’, like all 
the other student success indicators, is as much a matter of the classroom as it is of 
organised student life outside the classroom (Madiba 2014). The notion of student 
success is, therefore, not limited to what happens within the formal classroom. 
Student engagement is firstly defined as the amount of time and effort students 
spend on academic activities and other activities that lead to the experiences 
and outcomes that constitute student success. However, a second component to 
engagement is the way that institutions allocate resources and organise learning 
opportunities and services to induce students to participate in and benefit from such 
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activities (Kuh et al 2010). Because student engagement views student success as 
much broader than the classroom, it creates the space for intentional and collective 
effort by all role-players to optimise students’ chances of success. In fact, if student 
success, constituting learning and the development of graduate attributes make 
up the intended outcomes of higher education, we would argue that curriculum 
learning, academic development and student affairs need to combine forces and 
share expertise and resources to optimise students’ chances of success, as illustrated 


in Figure 7.4. 


Student engagement 


Student 
affairs 


Curriculum 
learning 


Student 
success 


Academic 
development 


FIGURE 7.4 Promoting intentional collaboration within a student engagement space 


We also suggest that student engagement could provide a framework and evidentiary 
base that give recognition to the contribution and role of lecturers, academic 
developers, student affairs professionals and students in creating environments 
that promote student success. South African student affairs can create innovative 
links between curricular and co-curricular learning, academic development and the 
holistic development needs of students. Through these partnerships, the knowledge 
base for each role-player in furthering the institutional mission to help students be 
successful could be broadened. For example, important perspectives that student 
affairs could provide on engagement and other data sources could be to assist 


institutions in: 
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= understanding the readiness of students to be engaged — connecting the 
developmental processes of students and institutional climate (socio-political 


context) with the ability of students to engage in the learning process; 


= facilitating environments outside the classroom that can promote engagement; and 


= providing greater engagement in academic learning by connecting the in-class 


with the out-of-class learning processes. 


Kuh (2011:259-267) also describes seven conditions that illustrate how student 


affairs can influence student success: 


l; 


Even though the identity formation of student affairs in the United States and in South 
Africa differs quite a bit, both recognise the important role these divisions could 
and should play in helping students learn and develop beyond the classroom. We 
contend that with newly established communication platforms, as well as datasets, 
such as the SASSE, student affairs divisions in South Africa can start building on 
evidence-based interventions that emphasise the importance of co-curricular 
support, development and learning for student success and holistic development. 


Position student success in the mission of student affairs and as an 
institutional priority. 

Facilitate student agency by teaching new students how to make 
good use of institutional resources. 

Collaborate with various institutional stakeholders to scale 
programmes or practices that have an impact. 

Establish and monitor early warning systems and safety nets to 
support students when they need help. 

Support academics to have a better understanding of the socio- 
political context of students so they are able to create a sense of 
community in the classroom and by connecting experiences inside 
and outside the classroom so that they are mutually reinforcing of 
holistic development. 

Focus assessment and improvement efforts on what matters to 
student success. 

Refocus the student affairs division to intentional, evidence driven 
facilitation of student success. 
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THE ROLE OF INSTITUTIONAL 
LEADERSHIP IN ADVANCING 
STUDENT ENGAGEMENT 


George Kuh & Paul Lingenfelter 


He who thinks he is leading and has no one following him is only taking 
a walk — Malawian proverb 


INTRODUCTION 


The world abounds with contestable assertions and equivocal data about the 
education policies and programmes that will prepare individuals to succeed and 
advance the greater social good. With this in mind, two truisms hold for postsecondary 
education today in every corner of the globe. 


First, the demands of life, work and civic responsibility require all graduates, no matter 
what their educational objective, to attain a certain level of proficiency with 21st 
century learning knowledge, skills, and dispositions. There are various descriptions 
of these outcomes including the essential learning outcomes (ELOs; see Figure 8.1), 
referred to as graduate attributes in other higher education systems, promulgated 
by the Association of American Colleges and Universities (AAC&U) (AAC&U 2008), 
the degree qualifications profile (DQP; see Figure 8.2), and the National Framework 
of Qualifications (NFQ) in Ireland (Quality and Qualifications Ireland [QQI] n.d.). 
While the descriptions of the outcomes featured in these statements may differ, they 
all generally agree that graduates should possess the major proficiency domains 
such as critical and analytical thinking, clarity and precision in writing and oral 
presentation, and a demonstrated capacity to apply and transfer what they know and 
can do to different situations. 


The implication of these and other outcome taxonomies is that simply holding a 
postsecondary certificate or a degree no longer automatically insures that a graduate 
can live an economically self-sufficient, satistying life. Rather, students must acquire 
knowledge and skills that will add value to their employers and communities, and they 
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must continue to learn in order to meet the demands of a rapidly changing world. If 
a student obtains a degree without acquiring these competencies, the degree won't 
be worth much. High-quality learning is no longer optional; it is required. 


The second truism is that for students to acquire the essential learning outcomes 
alluded to here, postsecondary institutions must be more intentional about helping 
students develop these capabilities and more analytical, self-critical, and adaptive 
in improving the effectiveness of teaching and learning. They must use engaging 
pedagogies that require students to invest substantial time and effort to acquire 
relevant knowledge and provide opportunities to practice applying what they have 
learned to the range of unscripted, messy problems they will encounter. The phrase 


The Essential Learning Outcomes 


Beginning in school, and continuing at successively higher levels across their college studies, students 
should prepare for twenty-first-century challenges by gaining: 
* Knowledge of Human Cultures and the Physical and Natural World 


= Through study in the sciences and mathematics, social sciences, humanities, histories, 
languages, and the arts 


Focused by engagement with big questions, both contemporary and enduring 


* = Intellectual and Practical Skills, including 
= Inquiry and analysis 
= Critical and creative thinking 
= Written and oral communication 
= Quantitative literacy 
= — Information literacy 
= Teamwork and problem solving 
Practiced extensively, across the curriculum, in the context of progressively more challenging 
problems, projects, and standards for performance 
* Personal and Social Responsibility, including 
= Civic knowledge and engagement—local and global 
= Intercultural knowledge and competence 
= Ethical reasoning and action 
"Foundations and skills for lifelong learning 


Anchored through active involvement with diverse communities and real-world challenges 


* = Integrative and Applied Learning, including 
= Synthesis and advanced accomplishment across general and specialized studies 


Demonstrated through the application of knowledge, skills, and responsibilities to new settings and 
complex problems 


FIGURE 8.1 Essential learning outcomes (AAC&U 2008) 
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commonly used to refer to the set of behaviours that are precursors to these outcomes 


is “student engagement” (Kuh 2001, 2003, 2009). 


The engagement premise is straightforward. The more students study a subject, the 
more they know about it. Similarly, the more students practice and get feedback from 
academics and support staff members on their speaking, writing and collaborative 
problem-solving, the deeper they understand what they are learning and the 
more adept they become at communicating effectively, managing complexity and 
ambiguity, and working with people from different backgrounds or who have different 
views. Participating in a variety of educationally productive activities, including what 
are known as high-impact practices (Kuh 2008), also contributes to the foundation 
of skills and dispositions needed to live a productive, satisfying life after university 


Degree Qualifications Profile 


*A template of competencies required for the award of college degrees at the 
associate, bachelor’s, and master’s levels 


Knowledge 


At each degree level, every college student should demonstrate competence in using both specialized knowledge 
from at least one field and broad, integrative knowledge from arts and sciences fields. Both kinds of knowledge 
should be pursued from first to final year, providing opportunities for integration across fields and application to 
complex problems—in the student’s area of emphasis, in out-of-school settings, and in civil society. 


Broad/Integrative Knowledge Specialized Knowledge 
Key areas include the sciences, social sciences, Students demonstrate depth of knowledge in a field 
humanities, arts, and global, intercultural and and produce field-appropriate applications drawing on 
democratic learning. both major field and, at the B.A. level and beyond, 
In each area, students: other fields. Students learn: 
Learn key concepts and methods of inquiry e Discipline and field-specific knowledge 
e Examine significant debates and questions e Purposes, methods, and limitations of field 
e Make evidence-based arguments e Applied skills in field 
In addition, at each degree level, students: e Integrative skills and methods drawing from 
e Produce work that integrates concepts and multiple fields and disciplines. 


methods from at least two fields 


Intellectual Skills 


Students hone and integrate intellectual skills across the curriculum, applying those skills both to complex 
challenges within major fields and to broad, integrative problem-solving challenges. Skills include: 


e Analytic inquiry 

e Information literacy 

e Engaging diverse perspectives 
e Quantitative fluency 

e Communication fluency 


Civic Learning 


Students acquire knowledge required for responsible citizenship both from their formal studies (see knowledge and 
skills, above) and from community-based learning, and demonstrate their ability to integrate both forms of learning 
in analyzing and addressing significant public problems and questions. Civic learning may be demonstrated through: 
research, collaborative projects and/or field-based assignments. 


Applied Learning 


Students demonstrate their ability to integrate and apply their learning (see knowledge and skills, above) in complex 
projects and assignments that may include: research, projects, practicums, internships, work assignments, 
performances, and creative tasks. 


“The Degree Qualifications Profile was commissioned by the Lumina Foundation following a series of national discussions about learning 
outcomes frameworks. It was released by the foundation as a beta version in January 2011 and is being tested in a number of grant-funded 
national experiments. 


FIGURE 8.2 Degree qualifications profile (QQI n.d) 
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study. In this sense, engagement helps students to develop a capacity for deep, 
integrative learning, which is the gateway to a lifetime of continuous learning and 
personal development (Kuh 2003; Kuh, O’Donnell & Reed 2013). 


This chapter addresses the key role of postsecondary leaders and managers at 
all levels in using information about student engagement to help promote and 
realise higher levels of student performance and institutional effectiveness. First, 
the chapter describes how the conceptions of the role of educational leadership 
and management have changed over past decades. This backdrop is important 
because it alters the nature of the responsibilities leaders have to their institution and 
students. The chapter then considers what leaders and managers can do to enact 
student engagement as an institution-wide strategy to induce students to take part 
in activities inside and outside the classroom that will help them attain the essential 
21st century learning outcomes. 


While postsecondary systems around the world differ in certain respects, our 
experience is that effective approaches to using student engagement data to improve 
teaching and learning are near-universal and can yield positive results when adapted 
appropriately to an institution’s context and culture. In this regard, while many of the 
examples used here are from colleges and universities in the United States, they can 
be adapted to institutions in other countries as well. 


THE “KNOWLEDGE SOCIETY” HAS RAISED THE BAR FOR EFFECTIVE LEADERSHIP 


In the 1960s, Fritz Machlup, Peter Drucker, and others pointed to the emergence of a 
“knowledge society”, — a world in which information and knowledge will increasingly 
become economically valuable. Their prescient observation was that knowledge 
and information were becoming comparatively more important than the historical 
sources of economic advantage, such as the land ownership, natural resources, 
efficient transportation, and even access to capital. In the half century since, vast 
changes in the world economy have been more dramatic than anyone might have 
imagined in the 1960s. In the last quarter of the 20th century, workers in the United 
States with only a high-school education watched their wages steadily fall and their 
unemployment rates grow. In 1964, four-fifths of jobs in the United States were held 
by people with a high-school education or less; by 2008, this educational group 
held only two-fifths (41%) of United States jobs. More than 30% of jobs were held 
by people with a BA or higher degree. In 2016, the three largest companies in 
the world by market capitalisation were based on information technology, Apple, 
Alphabet (Google), and Microsoft. wo more companies in the top ten, Facebook 
and Amazon, are internet based. 
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These striking changes also have occurred elsewhere in the world. In the 23 nations 
that lead the world in postsecondary educational attainment, more than 40% of 
the adult population aged 25 to 34 have a postsecondary credential. In the top 
six nations, more than half the population aged 25 to 34 have a postsecondary 
credential (OECD 2015). The importance of expanding higher education systems 
in African countries to generate knowledge that could address issues impacting the 
development of these countries is particularly stressed by the World Bank (Experton 
& Fevre 2010) and discussions around the United Nations’ post-2015 development 
agenda (United Nations Development Programme [UNDP] n.d.). The point is, even 
among countries that differ vastly in their developmental states, the importance of 
higher education as a vehicle towards a better life for all is constant. 


In his book, The World is Flat: A Brief History of the Twenty-first Century, Thomas 
Friedman (2005) described the forces that are spreading knowledge-based economic 
activity over the globe, and moving lower-skilled, lower-paying jobs to countries with 
lower wages and fewer knowledge-based workers. Although the off-shoring trend 
provides job opportunities in less-developed countries, the gains are temporary. As 
the quality of life improves in these countries, labour becomes more expensive, and 
low skilled jobs are eliminated by technology or move to still poorer countries. The 
long-term prognosis for low-skilled, decent paying work is very poor (Carnevale, 


Smith & Strohl 2010). 


Thanks to technological advances, learners now can hold a computer (cell and 
smartphones) in their hand. This has accelerated and deepened the impact of the 
knowledge society and dramatically raised the stakes for students. Simply acquiring 
information is no longer an advanced skill. A 21st century education requires 
knowing how to apply information to innovate and solve complex, unanticipated 
problems. It is now much more difficult for a person without higher levels of 
knowledge and skill to have an economically self-sufficient, satisfying life. As a 
result, increasing proportions of young people around the globe are pursuing some 
form of postsecondary education, and many older adults are returning to school in 
order to improve their economic prospects. 


The knowledge society has also raised the stakes for educators and those that lead 
and manage them. For example, in the 1950s and 1960s in the United States, 
plenty of students failed, but one never heard about “failing schools”. Today a 
Google search on the term “failing schools” produces 119 million results in less 
than one second. Unlike previous eras, schools now are expected to do more to help 
their students succeed. High rates of student failure are unacceptable, and it is no 
longer acceptable (or even possible for most institutions) to achieve success simply 
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by schooling only the academically talented and highly motivated. This places more 
pressure on educational leaders to improve institutional effectiveness. 


Even the most selective postsecondary institutions have a responsibility to enhance 
student learning, not simply to recruit brilliant people who can educate each 
other. Academically gifted, academically average, and capable but academically 
underprepared students all deserve to have teachers, mentors, and instructional 
approaches that support, challenge, encourage, and help them realise their full 
potential to lead satisfying, productive lives. 


Being prepared to participate actively in today’s knowledge economy is just the 
beginning. Students also need to understand the nuances of the contexts in which 
they live and work, learn how to interact effectively with people who think and act 
differently, meet the obligations of citizenship, and — as noted earlier — become adept 
at deep integrative learning, the gateway to continuous learning that will enable 
them to thrive in a rapidly changing world. These outcomes are the responsibilities 
of all educators. And it is the duty of leaders and managers to help create the 
circumstances necessary for this essential work. 


A NEW SLANT ON WHAT CONSTITUTES QUALITY IN TEACHING AND LEARNING 


The additional expectations and responsibilities placed on colleges and universities 
by the knowledge society demand different approaches to teaching and learning as 
well as different approaches to institutional leadership. For example, the marks of a 
successful university rector or president in the 20th century often included more and 
better qualitied applicants for admission as shown by entrance exam scores, greater 
selectivity reflected by larger numbers of students denied admission, and increased 
annual giving by alumni and friends. These are all measures of popularity and, to 
a degree, prestige. Popularity and prestige are not inherently bad, but productivity 
is better. In today’s knowledge society, the most effective institutions add value by 
helping their graduates acquire the needed knowledge and skills and increasing the 
proportion of students who fully realise their potential as learners. This means that 
institutional leaders at all levels must do more than attract and efficiently manage 
fiscal and human resources; they must also know how to deploy resources to improve 
student and institutional performance. 


In sum, shifts in pedagogical approaches and institutional leadership practices are 
needed to yield productivity increases in the quality of learning and the rates of 
academic accomplishment for all. It is beyond the scope of this chapter to delineate 
all the policies and practices needed to create these outcomes. However, two 


essential ingredients are essential for transformational improvements in teaching 
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and learning. The first is to clearly state the institution’s desired student learning 
outcomes. It simply is not possible to develop a meaningful strategy for improving the 
outcomes of university study without first specifying those outcomes. The second is to 
systematically obtain actionable student engagement data to identify where student 
effort is insufficient to attain the desired skills and dispositions and to use these data 
to guide institutional action. The rest of this chapter considers these two areas. 


ENGAGEMENT FOR WHAT? DELINEATING THE INSTITUTION’S INTENDED 
LEARNING OUTCOMES 


In the United States, the majority of postsecondary institutions have published, in some 
form, statements of their desired learning outcomes for all their undergraduates. 
Simply put, a student learning outcome (SLO) statement articulates what an 
institution wants students to get out of their university education. When crafted 
in understandable, actionable language, student learning outcome statements 
clearly and specifically express the knowledge, skills, attitudes, proficiencies, and 
dispositions that are commensurate with the desired degree or certificate. 


According to a 2016 policy statement from the National Institute for Learning 
Outcomes Assessment (NILOA), learning outcome statements are most useful when 
they represent outcomes valued by academic staff, students, and employers, and 
provide reference points for student performance for both individual courses and 
the cumulative effects of a programme of study. In this way, it is possible for students 
and others to see how the quality of their performance has improved over time. 
Equally important, student learning outcomes statements should inform educational 
policies and practices that should be moditied to guide improvement and stimulate 
pedagogical innovations. 


Good SLOs feature active, operational verbs to guide the design of assignments 
that motivate students to acquire and demonstrate the desired outcomes and enable 
others to verify their achievements. Clear, specific statements describing desired 
outcomes make it possible for academics to align curriculum and pedagogy with 
intended proficiencies, which is essential for ensuring that a programme is, indeed, 
achieving its purposes. Such statements also make it easier for students to understand 
and appreciate institutional and programme expectations for their performance and 
how their learning will equip them to handle what they encounter after graduating 


(Adelman 2015). 


Student learning outcome statements can be less helpful and problematic if they 
are intended primarily to meet compliance demands of external groups instead 
of helping academic staff achieve their own instructional goals. It might happen 
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that relatively few academic staff members have experience designing clear, explicit 
course and programme outcomes and crafting assignments that directly elicit the 
intended outcomes. Institutional leaders are responsible for creating the conditions 
in which academic staff are motivated and enabled to acquire these skills. 


This kind of collaborative work around engaging, outcome-linked assignments is 
required because NILOA surveys show that instructor-designed assignments are the 
primary vehicle through which students demonstrate that they know and can do 
what the institution or programme specifies. In the United States, such evidence 
is used far more frequently to inform and guide improvement than standardised 
tests. To underscore the critical role of assignments, NILOA has conducted a series 
of ‘charrettes’ in which academic staff from different fields discuss how to improve 
their assignments in ways that more accurately align with one or more intended 
proficiencies (Hutchings, Jankowski & Schultz 2016). Such assignments engage 
students in ways that induce them to expend more effort leading to higher levels of 
learning and personal development. 


So, what can institutional leaders do to promote higher levels of student engagement 
that lead to attaining the essential 2 1st century learning proficiencies and dispositions? 


LEADERSHIP TACTICS FOR ENHANCING STUDENT ENGAGEMENT 


At high performing higher education institutions, many people in a variety of 
positions collaborate with common purpose to challenge and support students to 
perform at high levels in order to meet the demands of the knowledge society (Kuh, 
Kinzie, Schuh & Whitt 2010). Shared action of this sort requires leadership and 
coordination from senior academic and student life staff to implement a strategy 
featuring student engagement as an institutional priority. Most leadership texts and 
experts advise that key personnel, starting with the chief executive officer (CEO) — 
rector or president — consistently remind academic staff and others about institutional 
priorities (e.g. Kuh et al 2010; Morrill 2007). Continuous reminders about institutional 
priorities are important for many reasons. Academic staff and students everywhere 
believe they are overextended. Different groups resonate to different approaches 
and use different language to communicate the same concepts. As a result, the CEO 
and other senior academic and student life leaders must frequently remind academic 
staff about the university’s commitment to use engaging pedagogies institution- 
wide, and make certain this commitment is featured prominently in the university’s 
strategic plan (Ehrmann & Bishop 2016). 


Because higher education institutions tend to be loosely coupled (Weick 1976) 
in terms of how various organisational units operate, a collaborative, collective 
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approach to leadership is usually more likely to be effective (Birnbaum 1992; Lillis 
2006), especially when it comes to enhancing the quality of the undergraduate 
experience (Eckel 2000; Felten, Gardner, Schroeder, Lambert & Barefoot 2016; 
Lillis 2006). In a collective leadership approach, it is possible and even desirable 
for a person with authority, such as a rector or president, provost, or dean to invite 
or challenge staff at different levels of the organisation to become actively involved 
in implementing an innovation (Kezar & Lester 2011). At the same time, effective 
leaders realise that to foster shared responsibility at all levels of the institution, a high 
degree of collaboration is required (Felten et al 2016). 


This type of collective leadership approach legitimates efforts to use engaging 
pedagogies throughout the institution, and encourages creative adaptations of 
interventions at the points of implementation. When practitioners are free to be 
creative and adaptive in implementing core principles, the innovation is more likely 
to have the desired effects and become institutionalised. 


In addition to these foundational but essential leadership behaviours, eight additional 
steps are needed to support and sustain the effective use of evidence to advance 
the student engagement agenda. Some of these steps can and should be taken at 
the outset, but others can only be fully realised over a longer term, several years or 
more. The first one is foundational to enacting both near- and longer-term goals. 


Find and keep the right senior academic officer 


Perhaps the single most important action a university can take to pursue and realise 
its educational priorities is to have an effective, right-minded senior academic 
administrator. Whatever the title, such as provost in a United States institution or 
(deputy) vice-chancellor academic in Australia or South Africa, this is the person 
who is second in command and whose job it is to maintain a laser-like focus on 
making sure staff are doing the right things to foster high levels of student and 
institutional performance. Because universities have many important tasks to perform 
and academic staff seem always to be in transition, stable senior academic and 
student life leadership is critical. No one is more important than the vice-chancellor 
academic/provost for providing coherence and tactical oversight in advancing the 
student engagement agenda. 


It takes about six or seven years for meaningful changes to take root in the institution’s 
culture and for student engagement policies and practices to become normative, or 
‘the way we do business here’. The right senior academic officer must be in place 
for at least this period to provide the continuity of vision and relentless leadership 
required to sew student engagement policies and practice into the institutional fabric. 
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This is why the president of California State University (CSU) Fullerton, Dr Millicent 
Garcia, aggressively and successfully sought out Jose Cruz to join her as provost. She 
knew of Cruz’ commitment to use data to inform institutional practice to help first- 
generation and low-income students succeed in university study. At CSU Fullerton, 
where these kinds of students are in the majority, Cruz and his colleagues have put 
into place programmes and activities based on student engagement to increase the 
first to second year persistence rate by about 10% over three years. 


Without a long-serving, right-minded, data-oriented, respected provost/vice-chancellor 
academic, it is almost impossible to create and institutionalise the other conditions 
that enable the effective use of student engagement data. 


Put someone in charge 


Although everyone should be responsible for student engagement, someone must 
be explicitly accountable for achieving it. Some individual or group must coordinate 
and monitor the status and impact of institutional student success initiatives. While 
unequivocal, vocal support from the provost and the president are important parts of 
the equation, their responsibilities are broad and too numerous to allow them on a 
day-to-day basis to monitor the progress of the student engagement agenda. Rather, in 
most instances a senior, well-respected academic staff member who has a reputation 
for getting things done should lead a student engagement task force or something 
similar. The general charge of the group should be to champion and periodically 
report on progress in improving student engagement. It could advocate and report 
on increases in the number of high-impact practices available to students and which 
students participate. It could also help interpret student engagement survey results 
and offer advice about other steps the institution can take to promote and support the 
use of engaging pedagogies. Sometimes a relative newcomer in such a group can 
lead the way — a new academic dean or student life officer with fresh ideas for ways 
to integrate students’ in-class and out-of-class experiences. ‘In charge’ parties are 
not necessarily expected to bring about the changes themselves, but to monitor, prod 
and support others who also were working on the issues. Other key resources can be 
teaching and learning centre staff and members of campus policy bodies. 


Insist on having and using current student engagement data to inform policy 
and practice 

Information about the way students spend their time is foundational to determining 
where to invest institutional effort and resources to improve student engagement. 
The National Survey of Student Engagement (NSSE) website offers substantive advice 
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for how often to collect this information and ways to analyse and report the results 
effectively, for example Lessons From the Field: Using Data to Catalyze Change on 


Campus (NSSE 2015). 


The Australian Council for Educational Research (ACER) provides additional examples 
from universities using the Australasian Survey of Student Engagement (AUSSE) 
(introduced in Chapter 6) as does the Irish Survey of Student Engagement (ISSE) 
administered through a partnership between the Ireland Higher Education Authority 
(HEA), several relevant representative institutions and organisations (Institutes of 
Technology Ireland, IOTI, and the Irish Universities Association, IUA), and the Union 
of Students in Ireland (United Statesl) (National Survey, Local Impact n.d.). The South 
African surveys of student engagement include examples of actions institutions can 
take in its annual report as well as web-based development facilitations through 
the Classroom Survey of Student Engagement (CLASSE) to improve teaching and 
learning for academics and academic developers. ' 


But administering engagement surveys is a waste of time if the provost or other 
academic leaders do not include student engagement information on the institution’s 
performance indicator dashboard, devote time at meetings of the institutional 
governing board and academic and student life deans discussing the implications of 


engagement data, and target resources to address shortcomings. 


A focus on student engagement may begin with verified data about graduation 
rates or time to degree and available information from direct measures of student 
performance on tests and instruments. At some point, however, the institution should 
also seek to discover as much as possible about post-university employment and 
post-graduate studies. 


Presidents, rectors, and other leaders can enhance an institution’s reputation by 
communicating many of the institutional metrics of student engagement to the wider 
world. In addition to metrics, institutional transparency should include a full and 
systematic account of the methods that are used to gather and act on evidence, 
such as in programme reviews at the departmental level that increasingly require a 
focus on student learning. The way evidence derived from these processes is used 
to change how courses are taught and programmes are improved can be described 
and illustrated. Senior academic and student life leaders can also support efforts to 
provide and interpret data that can be compared with similar institutions about critical 


1 The piloting and development of the first version of the South African Survey of Student 
Engagement (SASSE) took place in 2006. Chapter 1 discusses in detail the development of 
the surveys in the South African context. 
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aspects of student experience and attainment on everything from grading practices 
to success rates on professional qualifying exams. In today’s knowledge society, 
institutional leaders and those around them must stay focused on documenting and 


communicating the transformative power of education. 


Use student engagement data to develop more effective policies and practices 


As Richard Morrill (2007) advised, academic and student life staff must “close the 
empty space” between collecting data about student engagement and using it to 
inform and guide reforms. Too often, relevant information is sequestered in one 
office and not shared or connected with other information to create understandings 
that can persuasively mobilise action. Leaders must prioritise the use of data, and 
then make sure that the relevant curriculum and planning committees distil the 
implications born of the data and set forth action plans to address them. 


Presidents and senior staff also must work to ameliorate the negative effects of 
typical bureaucratic functions, such as evaluating, reporting and planning, driven 
by the compliance requirements of external entities. Without question, universities 
are obliged to be accountable to their stakeholders. But if these tasks are allowed to 
become an end in themselves, they will distract from the core business of educating 
students. Creative leaders can reshape the tools of accountability to become tools for 
institutional improvement. They can insist that data collecting offices and academic 
leaders work together to connect different data sources in ways that respond to 
accountability demands while featuring student and institutional accomplishments 
with an eye toward improving outcomes. 


Indeed, constructive staff involvement requires integrating student engagement 
data with the results of learning outcomes assessment principally for improvement, 
rather than for accountability. Ample evidence from education as well as from health 
care and business indicates that it is counterproductive to permit accountability to 
become the perceived purpose of assessment. When this occurs, all workers, and 
especially highly trained professionals, tend to become quite skilful in undercutting 
the validity of assessment indicators and subverting the effectiveness of potentially 
punitive accountability systems. To drive improvement, the staff members involved 
must fully accept the validity and importance of engagement indicators. If their 
personal values and motivations for personal effectiveness are reflected in these 
assessment indicators, academic and student life staff will be eager to use them for 
improvement. This is why whatever the source of student engagement data, whether 
it be from the South African Surveys of Student Engagement (SASSE) or locally 
developed tools, the information may not be useful unless it answers questions that 
academic and student life staff have about the student experience and how to more 
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often channel student effort into educationally purposeful activities. The good news 
is that we now have a better idea of how leaders can help their institution do this 
well. Kuh et al maintain that: 


= They involve the key stakeholders — academic and student affairs staff, students, 
governing board members, and others as appropriate at the beginning of any 
student engagement assessment project to determine the questions it needs 
to answer. 

= They build interest and momentum by creating occasions for people to work 
together to raise issues and questions they care for and need to know more about 
in order to improve student engagement and learning. 

= They reconvene these same people to make sense of the findings and tease out 
their implications for action. 

= They present engagement results in transparent, understandable forms to the 
people who have a need to know and act on them (Kuh ef al 2015). 


An instructive example is the University of Limerick where ISSE results were featured in 
specific briefings of the Vice-president Academic and registrar’s Management Group 
as well as in discussions of the Teaching and Learning Committee (a sub-group of 
Academic Council). ISSE reports were distributed to all faculties for discussion and 
the specific ISSE themes were aligned with the Student Evaluation of Teaching Survey. 
In addition, in February 2014, the university hosted a seminar on the ISSE as part of 
the Seminar Series funded by the National Forum for the Enhancement of Teaching 
and Learning. 


Modify the institutional incentive and reward systems to encourage and support 
the use of engaging pedagogies informed by student engagement results 


In order to enhance student engagement, a substantial number of academic staff 
must be attitudinally disposed to experiment with engaging pedagogies and practices. 
The combination of NSSE and Faculty Survey of Student Engagement (FSSE) data 
show that the greater the number of academic staff who believe it is important for 
their students to participate in high-impact practices (HIPs) before they graduate, the 
greater the number of students who participate in these practices. The main point 
here is not that the academic staff members themselves are involved in a high- 
impact practice, but rather that their attitudes toward HIPs and student engagement 
are positive enough such that they endorse the value of HIPs and encourage students 
to participate in such activities. 


At the same time, it is reasonable to assume that the more experience academic 
staff have with engaging pedagogies including HIPs, the more positive they will be 
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about student engagement in general and the policies and practices that support 
such efforts. In the School of Engineering at the University of Glasgow in Scotland, 
enhancing student engagement is key to an effort to promote student persistence and 
success by incorporating active learning, induction (or structured early socialisation 
activities) and participation in co-curricular activities (Browitt, Balance, Griffiths, 


Finlay & Rinaldi 2016). 


Instructional staff attitudes and behaviour are shaped by both the incentives built 
into institutional reward systems and the values to which staff have been socialised 
through their experience and academic training. It is unlikely that staff will experiment 
with and use different instructional approaches if the institutional reward system does 
not encourage and support efforts to improve student learning. If the criteria for 
contract renewal, promotion, and salary increases primarily or exclusively emphasise 
research, publication and scholarly recognition, teaching will receive short shrift. It 
is equally unrealistic, however, to expect teaching staff to respond to incentives for 
improving instruction that are not aligned with their fundamental understanding of 


and commitment to academic values. 


Thus, consistent messages from the president or rector, other academic leaders, 
and the governing board about improving student engagement and attainment are 
essential. However, such messages have a greater likelihood of encouraging and 
supporting behavioural changes when they are combined with concrete shifts in 
budgetary priorities and other forms of recognition that underscore and support high- 
quality learning and teaching. Presidents, provosts, and academic deans cannot set 
standards or establish meaningful incentives by themselves, but they have substantial 
influence in placing the issue on the agenda of the committees and other groups that 
periodically review faculty and student performance standards. In Making the Grade: 
How Boards Can Ensure Academic Quality, Peter Ewell (2012) argues that governing 
boards are key players in reinforcing and supporting the declarations by institutional 
leaders that focusing on increasing student engagement is an institutional priority, an 
academic duty, and a staff responsibility. As institutions gain more experience using 
evidence to improve teaching and learning, over time the patterns and priorities 
favoured by academic staff will change, especially when incentives and individual 
and programme recognition are linked to improved student performance. 


So, sooner or later (preferably sooner), tweaks will be needed to the institutional 
reward system and other incentives offered, such as small grants, to encourage and 
support experimentation with engaging pedagogies, HIPs, and other engagement- 
related efforts. Every institution has more than a few academic staff who are 
engagement savvy and routinely incorporate engaging pedagogies in their work 
with students inside and outside the classroom. But to grow the number who do so, 
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the reward system must materially acknowledge the importance and value of such 
work and should be represented in personnel policies and practices that encourage 
and support staff to experiment with engaging pedagogies and promising practices. 


Another way to reinforce the importance of student success is by publicly recognising 
excellence in teaching, research, or service at ceremonial events. For example, the 
University of Kansas bestows more than 20 teaching awards annually, many of which 
carry cash stipends. Many other postsecondary institutions in the United States provide 
such awards to acknowledge outstanding contributions to student achievement. 
Rewards and recognitions reinforce what is important and work best when they are 
transparent and operate in a manner consistent with espoused campus values. 


Cultivate an ethic of continuous improvement 


Typically, what is measured gets attention. Engaging universities publicly report on 
their performance and build feedback loops into the curriculum and other educational 
policies and programmes. People are characteristically ‘positively restless’; they 
are never quite satisfied with the status quo and are personally convinced that it is 
always possible to do better (Kuh et al 2010). They periodically and systematically 
review campus priorities, policies, and practices to ensure that what is enacted is of 
acceptable quality and consistent with the institutional priorities and values. Such 
examinations can be formal, such as programme reviews or accreditation self- 
studies, or less formal, such as departmental or programme-specific discussions 
about what is working well and what needs attention. The goal is to make data- 
informed decision-making a distinctive and prominent thread in the institution’s 
cultural fabric, so that everyone knows it’s ‘the way we do business here’. 


Scale up policies, programmes and practices that work 


Strong performing institutions typically have many high-quality programmes and 
practices. Moreover, they make certain that initiatives touch substantial numbers of 
students in meaningful ways, especially those students known to be at risk of leaving 
university prematurely. Indeed, one of the most persuasive warrants for increasing 
the use of engaging pedagogies and HIPs is that these activities have compensatory 
effects and, thus, have the potential to close the achievement gap. Thatis, participating 
in a HIP boosts the performance of students less well prepared for university study, 
those who are first in their family to go on to postsecondary education, and those 
who are from historically underrepresented racial and ethnic groups. 


Disaggregating student engagement results by various student background 
characteristics can reveal, for example, disparities in participation rates in HIPs. It 
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is, sadly, typical that students first in their family to attend university are less likely 
to study abroad, do an internship, or work on a research project with an academic 
staff member. As a result, they are less engaged overall in their studies and graduate 
at lower rates than their peers who do one or more of these activities. Leaders must 
demand that these data are evaluated every year and steps take to remove barriers 
to participation. 


Because of the desirable effects associated with participating in a HIP many 
institutions have or are in the process of making it possible for all students to have 
one or more of these experiences. One of the better documented stories about 
institutionalising engaging practices is that of Elon University as told by George Keller 
in his book, Transforming a College: The Story of a Little-Known College’s Strategic 
Climb to National Distinction (Keller 2004). It chronicles how the then president of 
Elon University, J. Fred Young, and his top aides in the 1990s used the literature on 
the conditions that enhance student development to devise an uncommonly wise 
strategic planning and institutional renewal effort to overhaul the curriculum to 
feature experiential, inquiry-based learning and cultivate a learning-centred campus 
culture. Compared with students at other United States colleges and universities, 
NSSE data show that two-thirds of Elon first-year students at least occasionally make 
class presentations compared with only 35% of freshmen elsewhere. Elon seniors are 
much more likely to study abroad (79%), do an internship or field placement (92%), 
and have a culminating senior experience (94%). 


Another example of institutionalising an engagement agenda is Hendrix College 
in Arkansas. All of its 1 500 students participate in its signature Odyssey Program, 
which like Elon features experiential learning in the form of three Odyssey activities 
(Hendrix College n.d.). These year-long experiences are similar in many respects 
to such HIPs as independent research or an internship in terms of what they induce 
students to do. As a result, Hendrix students demonstrate large increases between 
the first and senior year in the percentage of who participates in HIP-like experiential 
activities, placing it in the top 5% of all colleges in the United States. 


The mounting evidence of the power and value of experiential, high-impact practices 
persuaded the chancellor of the 64-campus State University of New York system to 
launch an initiative that requires every undergraduate student at every one of its 
two-year or four-year campuses to complete an applied learning experience during 
their studies. The 23-campus California State University has partnered with the 
National Association of System Heads in an effort to compile and share promising 
approaches to deploying engaging practices in an equity-minded manner to reduce 
the HIP participation disparities among students from different backgrounds. 
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At Ursinus College, all students complete the Common Intellectual Experience in 
the first year and an Independent Learning Experience, such as a research project, 
study abroad, or internship, before they graduate. All undergraduates at Miami 
University complete a capstone seminar. Georgia State University, San Diego State 
University, and the University of Texas at El Paso are among the scores of other 
institutions where institutional leaders over an extended period of time employed 
many of the tactics discussed in this chapter to inspire and support their colleagues 
to use engaging pedagogies to increase student accomplishment while reducing the 
achievement gap that bedevils many of their peer universities. 


The power of engaged learning and experiential education is a global phenomenon. 
For example, Tsinghua University in Beijing, China has adopted what is called the 
Maker Learning philosophy and approach for its undergraduate programme which 
features experiential learning in the form of peer-to-peer collaboration, flipped 
classrooms, and project-based activities. This combination is intended to foster 
design thinking through collaboration and integrative thinking (http://www.adelaide. 
edu.au/caustl/images/Thursday_ Zheng Li Keynote.pdf). The motivation for the 
institution-wide innovation is not unlike what is prompting similar efforts in the United 
States: how to reconfigure higher education so university graduates and those who 
employ them can compete more successfully in the global marketplace! 


Stay the course 


Universities do not become engaging institutions overnight. Leaders and other key 
actors focus their efforts for an extended period of time in order to establish them, 
demonstrate their efficacy, and sew them into the institutional culture in a manner 
that complements the institution’s espoused values and educational purposes. It is 
also the case that behind every engaging class or programme service are competent, 
caring people who share the institution’s vision for student success. Thus, one 
essential step to cultivate an engaging campus culture is for provosts, academic 
deans, and senior faculty to hire academic staff that are committed to enacting the 
student engagement agenda. 


Increasing student engagement institution-wide is not a linear or standardised 
process, but organic and experimental. Engagement results from a tool such as 
SASSE, which, if presented the right way can engender a spirit of innovation and 
continuous improvement in teaching, learning and gathering evidence of the kind 
that most academic staff value in their own scholarly work. Revelations about the 
value of student engagement come through academics working together to evaluate 
student performance on key courses and assignments, such as capstone courses 
and theses, or in the required gateway courses to a major, or on the learning goals 
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in general education courses and programmes. In this way, the institution gains 
experience working pragmatically across silos in order to solve problems and 


optimise educational opportunity. 


CONCLUSION 


There is no one best route to becoming an engaging institution. But the warrant for 
doing so is compelling. The 21st century places a high premium on sophisticated 
knowledge and skill, as outlined by the learning outcomes literature and associated 
frameworks (ELO, DQP etc.) mentioned earlier. Because the proficiencies and 
dispositions described in these frameworks are essential to an individual’s success 
and a productive national economy among other desired circumstances, institutions 
must take more responsibility for helping students acquire these capabilities. 
This means that institutions must be clear about their desired learning outcomes, 
assess the extent of their achievement, and use assessment to improve instruction 


and attainment. 


And this reminds us about why this book and institutional leadership are needed to 
successfully enact an agenda to foster higher levels of student engagement. Of course, 
students themselves must devote effort to educationally purposeful tasks to benefit 
from the resources institutions provide for their learning and personal development. 
At the same time, it is the institution’s responsibility to employ engaging pedagogies 


and high-impact practices to facilitate student engagement and achievement. 


To advance the engagement agenda, institutional leaders must constantly insist that 
higher levels of student attainment are a priority, while respecting staff leadership 
and giving staff opportunities to design approaches for improvement and measures 
of increased attainment that are congruent with the academic ethos. Leaders also 
must set forth the vision and provide sufficient support to enact policies, engaging 
pedagogies, and high-impact practices that are known to foster high levels of student 
engagement. They must also insure that processes are in place to measure and 
monitor student engagement in order to know what needs to be done to improve 


learning outcomes. 


Precisely how these conditions are created may vary, depending on the institution's 
educational purposes and culture along with the characteristics of its students. 
What must not vary is a firm, long-term commitment by institutional leaders and 
managers at every level of the organisation to ensure that all students reap the 


benefits of engagement. 
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PART II 


PROMOTING PEDAGOGICAL 
PRACTICES THAT MATTER 


Jillian Kinzie, Francois Strydom & Sonja Loots 


Education is what you know, not what's in the book — Egyptian proverb 


INTRODUCTION 


The focus on how students learn and what this means for instructional practice have 
expanded since Barr and Tagg (1995) declared that postsecondary education was 
undergoing a transition from an ‘instructional paradigm’ to a ‘learning paradigm’. 
The shift from an emphasis on delivering content to designing learning experiences, 
from thinking about inputs to focusing on outcomes, from higher education as a 
collection of separate activities to an integrated experience, has helped create a 
fertile environment for research about effective instructional practices. It has also 
helped encourage more educators to implement enriching educational opportunities 


that contribute to student learning and success. 


Coinciding with the shift to a learning paradigm, were concerns about improving the 
quality of undergraduate education and ensuring more students from wide-ranging 
backgrounds earned higher education degrees. These initiatives provided support 
for the establishment of United States assessment projects, namely the National 
Survey of Student Engagement (NSSE) and the Community College Survey of 
Student Engagement (CCSSE), which later also informed the development of the 
South African Surveys of Student Engagement (SASSE). These surveys promised valid, 
reliable and comparable data about educational quality by asking undergraduate 
students directly about their educational experiences and particular classroom 
activities that research shows enhance learning and success. The characteristics 
of student engagement are useful to measure because they impact directly on the 
quality of student learning and higher education outcomes. 
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Measurement of the extent to which students are engaged in practices that matter 
for learning provide an important opportunity to call attention to the presence or 
absence of the kinds of educational practices characteristic of a learning paradigm. 
If, as Shulman (2005:38) asserted, “learning begins with student engagement”, then 
methods for ascertaining the extent to which students are engaged in the learning 
process can offer helpful feedback to campus educators who wish to make an 
impact on student learning. With evidence from assessments of student engagement, 
practitioners concerned about student success can gain instructive insights about their 
students’ educational experiences and how it can be improved. For example, survey 
results can reveal the extent to which lecturers provide timely feedback on academic 
performance or which students are engaging in collaborative learning activities. Simple 
data points like these can catalyse discussions about expectations for feedback, course 
assignments and teaching practices (Kuh, Kinzie, Schuh & Whitt 2010). 


In the United States, higher education teachers’ interest in the learning processes 
of their students is growing (Doyle 2011), and a substantial body of knowledge 
exists on different research tested pedagogies to use and strengthen instruction 
and educational effectiveness (Ambrose, Bridges, DiPietro, Lovett & Norman 2010; 
Blaich, Wise, Pascarella & Roksa 2016; Mayhew et al 2016). As a result, this is 
an opportune time to consider all the ways enhanced instructional practice can 
positively influence undergraduate learning. In contrast, research on teaching and 
learning in South Africa has been overshadowed by a focus on transforming the 
broader post-democratic higher education system primarily through widening access 
and redressing gender and racial inequities (Department of Higher Education and 
Training [DHET] 2013). More recently, the focus has moved towards addressing 
the low student success rate through enquiries into student underpreparedness, 
curriculum adjustments, and the development of academic teaching and learning 
support. However, there is still a lack of context-specific literature about which 
educational behaviours effectively promote student engagement and student 
success. It is therefore also an opportune time to consider ways in which enhanced 
educational practice could impact student learning in South Africa, albeit for different 
reasons and in different developmental contexts. 


The chapter outlines the student engagement practices that matter for student 
learning and focuses specifically on those that instructors can influence. First, the 
chapter considers the foundational background and concept of student engagement 
within teaching and learning spaces, followed by a summary of research findings 
highlighting 19 student engagement and instructional practices that positively 
contribute to success. Next, the chapter turns to the South African context and 
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explores how effective pedagogical practices could be contextualised, and how 
or whether such practices impact academic achievement. The chapter concludes 
by presenting an integrated perspective to propose implications for pedagogical 
practice, with some emphasis on the South African context. 


STUDENT ENGAGEMENT AND TEACHING AND LEARNING 


The basic tenets of student engagement are rooted in research about effective 
instructional practice, particularly active and experiential learning. The Seven Principles 
for Good Practice in Undergraduate Education, developed by a group of scholars of 
higher education and distilled from 50 years of research, outlined a set of standards 
of active learning to apply to improve learning (Chickering & Gamson 1987, 1991), 
including (a) engaging in student-staff contact, (b) encouraging cooperation among 
students, (c) using active learning techniques, (d) providing prompt and meaningful 
feedback, (e) emphasising time on task, (f) communicating high expectations, and 
(g) respecting diverse talents and ways of learning. These seven principles are the 
basis of what is measured in the concept of student engagement and represent 
foundational instructional expectations for undergraduate education. 


This short list of empirically based, common-sense principles has been influential 
in broadening approaches to pedagogy on many campuses. The practices have 
been associated with many desirable outcomes, hold meaning for learning across 
professional programmes and the liberal arts, and foster learning among all students 
— including across racial-ethnic groups, ages, socio-economic statuses, genders, and 
levels of preparation (Cruce, Wolniak, Seifert & Pascarella 2006; Mayhew, Wolniak & 
Pascarella 2008; Pascarella, Salisbury & Blaich 2011; Pascarella & Terenzini 2005). 
Further, the principles assert the responsibility of educators and educational leaders to 
foster an environment favourable to good practice in higher education. 


The use of the term “student engagement” to encompass the seven principles followed 
from the conclusion that “the greater the student’s involvement or engagement in 
academic work or in the academic experience of higher education, the greater his or 
her level of knowledge acquisition and general cognitive development” (Pascarella 
& Terenzini 1991:616). In 2005, the authors concluded that higher education 
institutions should optimise conditions to encourage student engagement. Evidence 
showed that a key factor to whether students will survive and thrive once they enter 
higher education institutions is the extent to which they take part in educationally 
effective activities (Pascarella & Terenzini 2005). Most important about the student 
engagement concept is that it involves the intersection of student behaviours and 
institutional conditions. For example, high levels of purposeful student-staff contact 
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and active and collaborative learning supported by institutional environments 
perceived by students as inclusive and affirming can foster student engagement. 
Both student and institutional dimensions are critical to student engagement. 


IDENTIFYING PEDAGOGICAL PRACTICES THAT MATTER TO LEARNING 
AND SUCCESS 


The essential premise associated with student engagement is that the more students 
engage in practices that matter for learning, the more likely they will be retained, 
learn and persist to degree completion. To that end, a simple prescription for 
educators is to put as many student engagement practices in place in the curriculum 
and co-curriculum to maximise engagement. Of course, creating optimal student 
engagement experiences requires, for example, thoughtful pedagogical innovation, 
consideration of space and environment, collaboration with advisors, student affairs 
staff and other support services, and the creation of new or expanded enriching 
opportunities, for example, service learning or undergraduate research. Yet, willing 
educators equipped with information about the practices that matter most can 
certainly design a more engaging undergraduate experience. For example, knowing 
the specific student engagement practices that matter in the critical first year of 
study — a phase of the undergraduate experience with significant bearing on student 
success (Barefoot 2000; Upcraft, Gardner & Barefoot 2005) — is particularly useful 


given interest in improving completion and success. 


The exploration of practices that matter in the first year were highlighted in Unmasking 
the Effects of Student Engagement on First-Year College Grades and Persistence 
(Kuh, Cruce, Shoup, Kinzie & Gonyea 2008), a report of a large, multi-institution 
study that determined the relationships between key student behaviours and the 
institutional practices and conditions that foster student success. The study merged 
student-level records from students at different types of institutions to examine the 
links between student engagement and two key outcomes of higher education: 
academic achievement and persistence. 


With regard to the extent to which engagement during the first year of study has an 
impact on first-year grade point average and chances of returning for a second year, 
net the effects of student background, pre-college experiences, and prior academic 
achievement, this study confirmed that student engagement in educationally 
purposetul activities is positively related to academic outcomes as represented by 
first-year student grades and by persistence between the first and second study years. 
In these analyses, student engagement in educationally purposeful activities was 
represented as a summative scale of 19 NSSE items measuring student interaction 
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with lecturers, their experiences with diverse others, their involvement in opportunities 
for active and collaborative learning, and so forth. Each of the 19 items contributed 
equally to this global measure of student engagement and include: 


1) Asked questions in class or contributed to class discussions. 


2) Made a class presentation. 
3) Prepared two or more drafts of a paper or assignment before turning it in. 
4) Come to class without completing readings or assignments (reverse coded). 
5) Worked with other students on projects during class. 
6) Worked with classmates outside of class to prepare class assignments. 
7) Tutored or taught other students (paid or voluntary). 
8) Participated in a community-based project as part of a regular course. 
9) Used an electronic medium (chat, Internet, etc.) to discuss or complete 
an assignment. 
10) Used email to communicate with an instructor. 
11) Discussed grades or assignments with an instructor. 
12) Talked about career plans with an academic staff member or advisor. 
13) Discussed ideas from your readings or classes with academic staff members 


outside of class. 

14) Received prompt feedback from lecturers on your academic performance 
(written or oral). 

15) Worked harder than you thought you could to meet an instructor’s standards or 
expectations. 

16) Worked with academic staff members on activities other than coursework 
(committees, orientation, student life activities, etc.). 

17) Discussed ideas from your readings or classes with others outside of class 
(students, family members, co-workers, etc.). 

18) Had serious conversations with students of a different race or ethnicity than your 


own. 


19) Had serious conversations with students who differ from you in terms of their 
religious beliefs, political opinions, or personal values. 


These items were selected because previous research shows that all are positively 
related to desired outcomes of higher education (Pascarella & Terenzini 2005). 
Importantly, these items represent student behaviours and activities that institutions 
can influence to varying degrees through teaching practices and creating other 
conditions that foster student engagement. Using a multiple-item scale made the 
analysis more efficient and findings more reliable, since analysing the relationships 


191 


CHAPTER 9 © KINZIE, STRYDOM & LOOTS 


between large numbers of individual items can be misleading when applied to 
diverse groups of institutions and students. 


DETERMINING EFFECTIVE PEDAGOGICAL PRACTICES FOR THE SOUTH 
AFRICAN CONTEXT 


The same principle noted above, namely, that items representing effective 
pedagogical practices should represent student behaviours and activities that 
institutions can influence to varying degrees, as well as including a normative stance 
on developing socially conscious citizens who could contribute to social cohesion in 
South Africa, guided the selection of contextualised items from the SASSE. Drawing 
from Kuh ef al (2008), to identify which pedagogical practices impact on students’ 
academic achievement in the South African context, we expanded on the list of 19 to 
make room for contextualised practices and measured these items against students’ 
academic achievement. In the end, we identified 27 items representing three themes: 
Effective Educational Behaviours; Deep Learning; and Effective Teaching Practices. 
The following sections explain the processes of item selection, after which we report 
on how the contextualised items correlated with and explained variance in 1 985 
undergraduate students’ academic achievement. Because of privacy laws restricting 
the inter-institutional sharing of identifiable information, we only use institutional 
data from the 2014 SASSE administration at the University of the Free State, from 
where the student engagement surveys are administered. 


SELECTING EFFECTIVE PEDAGOGICAL PRACTICES 


The 19 items identified as effective educational practices were used as a conceptual 
starting point for developing contextualised effective pedagogical practices for 
the South African context. Five items were excluded from this list: three of them 
because we would like to distinguish between pedagogical practices (exclusively 
within classrooms) and high-impact practices (within or beyond classrooms, but 
primarily co-curricular learning). These items included acting as a tutor or peer 
learning facilitator; participation in community-based projects as part of courses; 
and working with staff members on projects beyond coursework. These items are 
included elsewhere in this book amongst developing high-impact practices for the 
South African context. The other two items we excluded from the initial 19 measured 
whether students used an electronic medium to discuss or complete assignments, and 
whether they use email to communicate with instructors. The first item was excluded 
because the institution makes use of a learning management system, which could 
impact the validity of the response to this item; and the second item was excluded 
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because many students do not have access to computers or internet when they are 
not on campus, which would also skew data for this item. Lastly, although there is 
significant overlap between the NSSE and SASSE items, some items are phrased 
differently or merged with other items to increase their relevance and understanding 
amongst South African students and staff. That means that the first set of items based 
on the original 19 items (and excluding the five mentioned earlier) focusing on 
Effective Educational Behaviours, consists of 11 items. 


Considering the national importance placed on redressing social injustices in South 
Africa and the prominent role higher education should play in shaping reflective 
and socially conscious citizens (e.g. DHET 2013), we learn from Nelson Laird, 
Shoup, Kuh and Schwarz’s (2008) findings that students who engage more in 
deep approaches to learning experience higher levels of personal and intellectual 
development, as well as more satisfaction with their college life. Further, drawing 
from research done by Nelson Laird, Shoup and Kuh (2005) to develop the NSSE 
Deep/Integrative Learning Scale, as well as drawing from items relevant to the social 
cohesion responsibilities prioritised by national government, we developed the Deep 
Learning theme, consisting of 11 items. 


Finally, items measuring effective teaching strategies were combined to form the 
third theme of Effective Teaching Practices. In the end, the 27 items looked as such: 


Theme 1: Effective Educational Behaviours 


1) Asked questions or contributed to module/subject discussions in other ways. 

2) Gave a module/subject presentation. 

3) Prepared two or more drafts of a paper or assignment before handing it in. 

4) Attended class without having completed readings or assignments. 

5) Discussed your academic performance with a lecturer. 

6) Talked about your career plans with a lecturer. 

7) Discussed module/subject topics, ideas, or concepts with a lecturer outside of class. 
8) Asked another student to help you understand module/subject material. 

9) Explained module/subject material to other students. 

10) Worked with other students on projects or assignments. 


11) During the current academic year, to what extent have your modules/subjects 
required you to do your best work? 
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Theme 2: Deep Learning aimed at supporting socially conscious graduates 


12) During the current academic year, how often have you had discussions with: 
people of a different race or ethnicity than your own? 


13) People from an economic background different from your own? 

14) People with religious beliefs different from your own? 

15) People with political views different from your own? 

16) Combined ideas from different modules/subjects when completing assignments. 
17) Connected your learning to societal problems or issues. 


18) Included diverse perspectives (political, religious, racial/ethnic, gender, 


economic, etc.) in module/subject discussions or writing assignments. 
19) Examined the strengths and weaknesses of your own views on a topic or issue. 


20) Tried to better understand someone else’s views by imagining how an issue 
looks from his or her point of view. 


21) Learned something that changed the way you understand an issue or concept. 


22) Connected ideas from your modules/subjects to your prior experiences 
and knowledge. 


Theme 3: Effective Teaching Practices 

23) Received detailed feedback shortly after you completed tests or assignments. 
24) Clearly explained module/subject outcomes and requirements. 

25) Presented module/subject sessions in an organised way. 

26) Used examples or illustrations to explain difficult points. 


27) Provided feedback on a draft or work in progress. 


DETERMINING THE RELATIONSHIP BETWEEN EFFECTIVE PEDAGOGICAL 
PRACTICES AND ACADEMIC ACHIEVEMENT 


Because the higher education system in South Africa does not work with official 
Grade Point Average scores, a similar average mark from all final examination 
marks in 2014 was calculated for each student. A multiple regression analysis was 
then done to determine which of the specific items correlate with students’ marks, 
as well as whether the individual three themes and set of 27 items could predict a 
variance in students’ academic performance. 
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Out of the 27 items, ten items had a significant positive correlation with the whole 
sample of students’ marks. The respective items, along with the Pearson correlation 
coefficient (r), the number of participants included in the analysis, as well as the level 
of significance are shown in Table 9.1. 


TABLE 9.1 Correlations between 27 contextualised educational practices and academic 
achievement 


Theme 1: Effective Educational Behaviours Pe fn] oe | 


Asked questions or contributed to module/subject discussions 


j .095 1977 .000* 
in other ways 


Attended class without having completed readings or 


: i .074 1962 .001* 
assignments (Reversed scoring) 


Talked about your career plans with a lecturer .046 | 1978 | .040** 


Explained module/subject material to other students .102 | 1959 | .000* 


During the current academic year, to what extent have your 
modules/subjects required you to do your best work? 


.071 1872 .002* 


Theme 2: Deep Learning aimed at supporting socially 
conscious graduates 


During the current academic year, how often have you had 


discussions with: people of a different race or ethnicity than 1969 | .002* 
your own 
| People from an economic background different from your own | .068 | 1965 | .002* 


Theme 3: Effective Teaching Practices r PN |e | 


Received detailed feedback shortly after you completed tests or 


f .046 1950 .042** 
assignments 


Clearly explained module/subject outcomes and requirements | .106 | 1961 .000* 


Presented module/subject sessions in an organised way 084 | 1947 | .000* 


*p<.01; **p<.05 


While all the positive correlations between items and marks are statistically significant, 
the correlations themselves are not very strong. For example, the two items with 
the strongest correlations only show correlation coefficients of r=.102 and r=.106 
respectively. 
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Four multiple regression analyses in the form of four models (1=Theme 1; 
2=Theme 2; 3=Theme 3; 4=total of 27 items) were then tested to determine the 
extent to which these themes impact on the variance in students’ end-of-year marks. 


TABLE 9.2 Multiple regression analysis: total UFS sample 


df df Si R2 R2 
regression | residual 9 adjusted 


Model 1 1831 .000* .036 .031 
Model 2 2.071 11 1908 .020** .012 .006 
Model 3 6.830 5 1916 .000* .018 015 
Model 4 4.512 27 1806 .000* .063 .049 


*p<.01; **p<.05 


Table 9.2 shows that each of the individual themes (Effective Educational Behaviours; 
Deep Learning; and Effective Teaching Practices), as well as the 27 items as a whole 
contribute significantly to explanations of variance in students’ marks (all p<.05). 
In statistical terms, the adjusted R square values used to determine the extent of 
this impact seems relatively low. For example, Theme 1 is responsible for 3.1% of 
variance in marks; Theme 2, for 0.6%; Theme 3 for 1.5%; and the total group of 27 
items is responsible for 4.9% of variance. However, considering that we are applying 
statistical methods to educational settings outside of a controlled environment, 
where human complexities are unpredictable, a 5% variance in marks could count 
for the difference between acceptance into a programme or rejection; between 
reassessment or pass; or between a regular pass and a distinction. 


Figure 9.1 shows a visual representation of how students’ average marks increases 
alongside an increase in each of the educational themes. 
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FIGURE 9.1 Average marks compared to educational theme scores 


Students’ average marks of 54.83 increases by 6.48 the higher they score on 
Theme 1 (Effective Educational Behaviours) items. The second highest impact on 
marks is Theme 3 (Effective Teaching Practices), where students’ average marks of 
56.29 increase by 3.59 as they score higher on the theme’s items. Lastly, Theme 2 
(Deep Learning) shows a very slight increase of 0.38 points between those scoring 
lower and higher on the theme’s items. 


WHAT THE DATA CONTRIBUTE TO OUR THINKING ABOUT EDUCATIONALLY 
EFFECTIVE PEDAGOGIES 


The data presented here provide a powerful starting point for reflection about 
teaching and learning practices, and the broader pedagogical environments they 
take place in. The most significant finding is that effective pedagogical practices 
matter — particularly when combining student effort or agency, deep learning 
practices and classroom environments. For the whole sample, these practices 
accounted for almost 5% of variation in academic achievement. Other important 
findings include: First, the correlational analyses highlight effective educational 
behaviours — many of which students are in charge of — impacting on academic 
performance. In other words, when students ask questions in class, when they go to 
class prepared, when they go and talk to their lecturers about possible career plans, 
when they explain things to other students, and when they find the academic content 
challenging, they tend to perform better academically. Correspondingly, lecturers 
must create opportunities for students to exercise these behaviours. Students are 
more likely to ask questions when assignments and in-class exercises demand active 
engagement, and when peer teaching and group learning are carefully structured 


197 


CHAPTER 9 © KINZIE, STRYDOM & LOOTS 


in courses. Secondly, the data tell us that contact with diversity influences academic 
achievement. That is, broadening one’s mind to include perspectives of others who 
experience life differently — in this case from a different racial perspective or from a 
different socio-economic perspective — impacts on academic achievement. Finally, 
the data tell us that classroom environments matter. Students’ marks are positively 
influenced when they receive detailed feedback shortly after assessments, when 
subject outcomes and requirements are explained clearly to them, and when subject 
content is presented to them in an organised way. These items imply that when 
students know what is expected of them and they can follow the logic of the course, 
they perform better academically. 


This leads us to question what it means that many items do not show significant 
correlations with students’ academic achievement. We suggest there are two 
reasons: First, these behaviours are not practiced to the extent that they should be to 
impact academic achievement; and second, the quality of the practices might need 
to be improved. 


Regarding whether practices take place, there is enough student engagement data 
presented in different chapters in this book from different sources — students, lecturers 
or both — to get an idea of what happens (or doesn’t happen) in classrooms. For 
example, the current analyses show us that the engagement subscale of reflective 
and integrative learning, representing seven of the 11 items in the Deep Learning 
theme, does not significantly contribute to students’ academic achievement. Yet, 
we know from student learning literature that the ability to integrate, synthesise, 
analyse and make judgements about information is key to building knowledge 
and understanding about subject matter (e.g. Ambrose et al 2010; King & Baxter 
Magolda 2011). We also know that even though these skills are valued by both 
students and lecturers, lecturers primarily rely on traditional lecturing techniques to 
transfer knowledge as opposed to making use of interactive classroom techniques 
(see Chapters 4 and 10). This, combined with the lack of relationship between Deep 
Learning and academic achievement, point to a problem in the system between 
intentions/values and practice. The lack of skills developed through Deep Learning 
also impacts on the success of our graduates, which extends beyond merely passing 
subjects. It impacts on students’ employability, their ability to contribute to the 
country’s social cohesion, and even their ability to pursue postgraduate studies, 
where these skills are essential. 


If effective educational practices do take place, it is important to gauge the quality of 
these practices. For example, we know from Chapter 5 and Chapter 10 that students 
often participate in group work, and that besides lecturing, group discussions are 
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favoured by lecturers (Chapter 4). If we know that a lack of implementation is not 
the issue, why then do we not see a relationship between working with other students 
on projects or assignments and academic achievement? We believe the answer 
here lies in the quality of the practice. It is not enough to merely divide students 
into groups and have them work on an assignment or discuss an issue. To have 
students optimally benefit from group interactions consideration needs to be given 
to diversity in groups, student buy-in, allocating responsibility and consequences, 
claritying expectations and outcomes, etc. 


SO, WHAT DO WE TAKE FROM THIS? 


From the perspective of teaching and learning it is noteworthy that these analyses 
show student engagement in a range of behaviours that institutions can readily 
influence through a variety of common teaching practices can make a significant 
difference in student learning and success. It has been shown that common 
practices that have been tried and tested in the United States including active and 
collaborative learning, making class presentations, and preparing multiple drafts of 
an assignment, positively affect grades in both the first and last year of study as well 
as persistence to the second year at the same institution, even after controlling for a 
host of pre-college characteristics and other variables linked with these outcomes, 
such as scholarships or merit aid and parental education. 


The list of pedagogical practices that matter reveals several practices that research 
and volumes about effective teaching have long declared as important to learning: 
a) frequent, timely feedback, b) active learning by asking questions and making 
presentations, and c) collaboration with peers. By all accounts, students need to be 
actively engaged in their courses through small group discussions, one-minute papers, 
quizzes, think-pair-share activities, and other common techniques (see Chapter 10 
for more detail on developing classroom techniques). Quite simply, lecturing, when 
combined with active-learning techniques, is a worthwhile alternative to a lecture- 
only format. The need for early and frequent feedback, particularly when coupled 
with opportunities for revision, helps students learn from their mistakes. Though 
feedback can seem time-consuming, if done strategically and with opportunities 
for revision or when combined with peer feedback, it can save lecturers time and 
improve the quality of student work. In fact, peer reviews can enable students to 
feel empowered by the process of giving and receiving comments and can lead to 
improved writing versus receiving feedback from the instructor alone. Increasing the 
frequency of student engagement in these three core pedagogical practices alone 
would make a significant difference in student success. 


199 


CHAPTER 9 © KINZIE, STRYDOM & LOOTS 


Because peers are so very influential to student learning and persistence (Astin 
1977; 1993), institutions must harness and shape this influence so it is educationally 
purposeful and helps to reinforce academic expectations. Notably, several of the items 
on the lists of pedagogical practices that matter involve peers as both collaborators 
and teachers. An important finding from the South African data is the relationship 
between explaining subject material to others and academic achievement, which 
could impact the way we think about collaborative learning inside and outside 
classrooms. Well-designed, in- and out-of-class group assignments and projects, 
facilitated study sessions, and coordinated discussion groups and recitation sections, 
are critical to structuring into the undergraduate experience, and particularly the 
first year, to ensure students build valuable peer support networks. Academic staff 
members should design more collaborative learning activities in-class, and also 
require more such projects and assignments. Intentionally structured occasions that 
bring students together for meaningful educational tasks in and out of the classroom 
are important opportunities for engagement. 


Several forms of interaction with staff, including talking with staff about marks and 
assignments, discussing career plans, communicating via email, and working with 
lecturers on activities outside of courses, are also significant to student success. 
These educational practices are common, academically supportive interactions, for 
example, discussions to clarify the demands or direction for an assignment. However, 
they are also important opportunities to demonstrate lecturers’ approachability and 
helpfulness and to ensure students feel a sense of support from academic staff. 
These interactions also provide occasions for career discussions, and some goal 
setting and planning. These could be performed by a staff member or advisor, but 
having someone to help set academic goals and create a plan for achieving them is 
an important part of students’ early academic success. In general, the more students 
report having these interactions with lecturers the more support they feel for their 
learning and belonging. 


Of course, whatever the contact with staff, it should represent a balance of challenge 
and support. Staff who make themselves available for office hours and discussions 
about assignments must display genuine support for students and affirm students 
as learners. Students who are unaccustomed to talking with lecturers, or who are 
in academic difficulty and are meeting with lecturers with some reluctance, might 
need greater encouragement and guidance about how to take advantage of these 
interactions. Peer tutors, advisors and student affairs and support staff can play an 
important intermediary role to facilitate students’ interaction with academic staff, 
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encouraging reticent or underprepared students and helping them make the most 
of the experience. 


The classroom is the most common venue that most students have for interacting 
with other students and with staff (Tinto 2000, 2001). Thus, using the classroom 
to create communities of learning must be a high priority in terms of creating a 
success-oriented campus culture. This means that staff members must also be more 
intentional about teaching institutional values and traditions and informing students 
about campus events, procedures, and deadlines such as registration. Creating a 
classroom experience that acknowledges campus life experiences is an important 
approach to fostering students’ connections to campus. Staff members in partnership 
with student affairs professionals and other staff familiar with culture-building 
strategies can work together to create a rich, engaging classroom experience that 


complements the institution’s academic mission. 


Turning to the South African context more specifically, even though the proposed list 
of educationally effective pedagogical practices was only tested on one university 
sample for the purposes of this chapter, it paves the way for important discussions 
on how we can identify which practices matter in student learning and how we can 
promote these practices to support student success. 


IN CLOSING 


Edgerton (2001) introduced the term “pedagogies of engagement” to describe 
the modes of teaching and learning needed to improve educational quality. The 
teaching and learning practices named in this chapter provide a ‘checklist’ of sorts 
for staff who wish to increase practices that matter for student learning and success. 
However, even more specific models for understanding student engagement from 
a pedagogical standpoint are available. For example, Barkley (2010) developed 
a classroom-based model for understanding student engagement emphasising 
engagement as both a process and product of the interaction between motivation 
and active learning. She outlines an array of instructional practices that can be set 
up by lecturers to create conditions that enhance students’ eagerness to learn and 
require students to do more of the work. Barkley profiled lecturers who created 
such conditions, and provides a large collection of student engagement techniques 
developed from good teaching practice literature. 


One of the more promising findings from the NSSE and CCSSE projects is that the 
effect of engagement is greater for historically underserved students including those 
who are underprepared, are students of colour, or are the first in their family to go 
to higher education. Essentially, engagement is a powerful tool that can facilitate the 
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retention and graduation of students, especially those from underserved groups. This 
evidence has compelled institutions to seek ways to channel student energy toward 
educationally effective activities, especially for those who start higher education with 
two or more debilitating factors — being academically underprepared or first in their 
families to access higher education, or from low income backgrounds. 


In fact, several scholars have explored the value of engagement for working with 
underprepared students. Gabriel (2008) outlined techniques that lecturers can 
employ to help students raise their skills. Prominent in her principles for instruction 
is that teachers must help students believe in their ability to learn (Dweck 2000), 
and that by modelling, providing feedback, talking to students about their effort and 
how they can take advantage of academic support resources, lecturers can enhance 
students’ learning and success. Designing engaging active learning strategies, 
including opportunities for frequent feedback and substantive peer collaboration 
experiences, are effective educational practices for all students, but particularly for 
underprepared students who are most in need of lecturer support and guidance. 


Greater attention to educational quality calls for lecturers to explicitly consider how 
students engage in their higher education experience and to create learning activities 
that are in harmony with findings about the best ways to help students learn and persist 
in their studies. As mentioned earlier, to create optimal engagement experiences 
in classrooms will require thoughtful pedagogical innovation, consideration of 
space and environment, collaboration with advisors, student affairs staff, and other 
support services. With evidence of the importance of student engagement, educators 
concerned about student success can gain instructive insights about their students’ 
educational experiences and how they may be improved. Findings from student 
engagement projects point squarely to pedagogical practices that lecturers can 
enact to make a difference to student learning. We hope through the handful of 
engaging practices discussed in this chapter to stimulate more research and debates 
on how we can motivate students to be active agents in their own learning; how we 
can create curricular and co-curricular spaces conducive to student learning and 
development; and ultimately how we can encourage the development of successful, 
well-rounded graduates through our engagement work. 
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USING ENGAGEMENT DATA 
FOR CHANGE AND 
EMPOWERMENT AT 
COURSE LEVEL 


Francois Strydom & Lana Hen-Boisen 


He who is wise endeavours to learn how to understand the truth not less 
than that — African proverb 


INTRODUCTION 


The teaching and learning relationship is crucial to high-quality student learning, 
personal development and success during and after university. Creating optimal 
classroom environments to support good teaching practices is therefore pivotal 
and requires that lecturers adopt a student-centred pedagogical methodology 
(Ouimet 2011). However, even though a significant body of knowledge exists 
related to teaching and learning, what happens in classrooms between lecturers and 
students is not often captured using data except for student evaluation of instruction 
forms, which is often required as part of institutional quality assurance but seldom 
intentionally used to further improvement. 


This chapter illustrates how course/module level student engagement data from the 
Classroom Survey of Student Engagement (CLASSE) can be used to create effective 
teaching and learning environments and promote evidence-based improvement in 
pedagogical practices. The chapter shows how the data can give students a voice 
(agency) in how they are taught, how the data can be used to empower academics 
to be better teachers, and how evidence can help to facilitate more collaborative 
relationships between students and their teachers. The chapter further explores 
areas where students and staff tend to ‘miss’ each other the most; as well as the 
implication of the use of this measure for an evidence-based approach to academic 
staff development. 
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WHY FOCUS ON THE COURSE/ MODULE LEVEL? 


Chapter 1 of this book highlighted a range of institutional level and course/ 
module level student engagement surveys. Institutional level surveys, in the South 
African context, include the South African Survey of Student Engagement (SASSE), 
the Beginning University Survey of Student Engagement (BUSSE), and the Lecturer 
Survey of Student Engagement (LSSE), which have been explored in different 
chapters of this book. The institutional level measures provide powerful data of 
an aggregated whole, which can be used for management level decision-making 
about the allocation of resources to different student groupings within the institution. 
However, disaggregating these large datasets to inform decisions on a micro-level 
in departments and modules runs the risk that sample sizes become too small or 
even non-existent, and secondly, staff and lecturers may protest against negative 
institutional level measure results with a defensive refrain: “these are not my students”. 


The CLASSE was developed by Judith Ouimet and Bob Smallwood (2005). The 
measure aims to collect data on levels of student engagement in a single module/ 
course based on the premise that classroom-level insights about the quality of 
student engagement can aid institutional efforts to enhance the adoption of 
engaging educational practices as well as guide the professional development of 
academic staff. CLASSE consists of a pair of survey instruments administered among 
students (CLASSE, saen) and the lecturer(s) (CLASSE 
CLASSE, sae data offer quantitative information on the time spent and frequency of 
engagement on educationally purposeful activities. CLASSE 


of a specific module. 


Lectu a 


lecturer data allow lecturers 
the opportunity to reflect on how important they consider these effective educational 
practices to be in their module in order for the students to be successful in that class. 
Therefore, CLASSE data can be used to improve teaching and learning practices, 


with the goal of improving student success rates. 


The most powerful way to analyse and present the CLASSE data is by comparing 
the responses of students and lecturers to identify the importance of an educational 
practice as well as the frequency with which students are participating in the relevant 
practice. Of importance are the educational practices that lecturers perceive as 
important but that students do not engage in to the satisfaction of their lecturers. 
Additionally, the data allow staff to encourage students to participate in certain 
educational practices and by doing so, positively impact engagement. Staff may 
be able to evaluate and reflect on their teaching styles and adjust them accordingly 
to expose students to activities that can improve their learning efficiency (Ouimet 
& Smallwood 2005). CLASSE results can also help identity effective educational 
practices, shape teaching and learning experiences, and inform departmental and 
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institutional development initiatives. The combined use of institutional level measures, 
such as the SASSE and the LSSE, with a course/module level measure (CLASSE) can 
provide data that can help the promotion of student engagement on multiple levels 
of the institution. SASSE and LSSE comparisons enable student and staff perspectives 
to be contrasted at the institutional level, while CLASSE enables reflection on the 
differences between lecturers and the students at the module level. 


EXPLORING THE CLASSE INSTRUMENTS 


In the South African context, the CLASSE surveys are designed and administered 
through online survey-software and are often used as a diagnostic measure in 
modules/courses with high failure rates. The surveys are completed within four to six 
weeks after the start of the module class-sessions. It is important to allow enough 
time for students to become accustomed to the module and its presentation to form 
an accurate opinion before the survey is administered. CLASSE is complimentary to 
any other survey or institutional data and can be used in conjunction with the data 
received from module evaluations, as CLASSE does not at all ask any questions 
related to the lecturer as an individual, neither does it evaluate the competence of 
the lecturer(s). Table 10.1 provides an overview of the CLASSE surveys. 


TABLE 10.1 CLASSE subscales and number of items 


Part | Engagement Activities 21 
Part Il Cognitive Skills 5 
Part Ill Other Educational Practices 10 
Part IV Class Atmosphere 4 
Part V Supplementary Learning Activities 5 
Part VI Demographics 9 


As shown in the table, the CLASSE surveys are each made up of 54 items divided into 
six sections. Since CLASSE is adapted from SASSE, items foreground engagement 
practices, particularly based on the work by Chickering and Gamson (1987) and 
Krathwohl (2002). 


tems measuring Engagement Activities include matters such as class participation, 
and collaborative learning. Cognitive Skills include synthesis, analysis and practical 
application of theoretical knowledge, while Other Educational Practices and 
Class Atmosphere focus on behaviours and classroom environments respectively 
that have been proven to be conducive to learning. The five items grouped under 
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Supplementary Learning Activities ask about experiential learning, service learning, 
fieldwork, laboratory work and clinical teaching. 


In addition to the standard 54 items, CLASSE also provides the lecturer of any given 
module the opportunity to add up to eight items. Because each module is different 
and includes unique facets, modes of administration and additional support services, 
the eight additional items allow for greater accuracy as a diagnostic tool. 


CLASSE RESULTS 
The CLASSE results are provided to lecturers in three sections, namely: 


= Student respondent characteristics; 
= Frequency distribution for student and lecturer responses; and 


= Quadrant analysis. 


The respondent characteristics report provides details of the demographic profile 
of the CLASSE student sample. Reflecting on student demographics provides an 
opportunity for lecturers to consider the varying contexts of all students in their class. 
Except for the visible demographics, such as race and gender, lecturers are provided 
with information on their students’ home language, age, repeater status and whether 
any visible or invisible disabilities are present in the student cohort. The frequency 
distribution for the CLASSE report is based on the responses of the students on all 
the student engagement items in a specific module. The number of students, and 
the corresponding percentage of the sample, is given for each item. The frequency 
of participation in various educational practices reported by the students can be 
compared to lecturer(s) ratings to determine whether students participate satisfactorily 
in educational practices that the lecturer considers to be important. 


The quadrant analysis provides a comparison of the importance rating provided 
by lecturers and the frequency rating provided by students in order to establish 
discrepancies or areas of agreement between lecturers and students. A comparison 
of the answers from students and staff points out educational practices that staff 
perceive as important but are being not being performed by students as often as 
desired. Staff can then have grounds on which to encourage students and positively 
affect engagement. 


The most powerful section of the CLASSE report is the quadrant analysis, shown 
in Figure 10.1. The CLASSE data displayed in the quadrant analysis can be used 
to facilitate a conversation about where students and staff miss each other. The 
vertical axis displays the lecturer(s) responses regarding the importance of certain 
effective educational practices and behaviours, while the horizontal axis shows the 
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aggregated frequency at which students experience and engage in these practices 
and behaviours. 


a 3 
g Quadrant Analysis | 
QUADRANT 2 QUADRANT 1 
Very Important or Important to Lecturer Very Important or Important to Lecturer 
Below Average Student Frequency Above Average Student Frequency 
Oppurtunity for improvement If a lecturer thought an item was 
important and students rated it as 
Example: Worked with other students on occuring with above average frequency it 
projects/assignments during this module would be shown in this quadrant. 


The educational practices that lecturers 
point out as being important or very 
important to them but that students 

report participating in at below average 

frequency will appear in this quadrant. 


QUADRANT 3 


Somewhat Important or Not Important to 
Lecturer Below Average Student Frequency 


QUADRANT 4 


Somewhat Important or Not Important to 
Lecturer Above Average Student Frequency 


Important (lecturer ratings) 


Items that fall in this quadrant are items 
that lecturers value as somehwhat 
important to not important and that 
stdents report participating in an above 
average frequency. 


q Frequency (student ratings) J 


FIGURE 10.1 Quadrant analysis 


If a lecturer rated an item as somewhat 

important and students reported that it 

occured at below average frequency it 
would be placed in this quadrant. 


The two upper quadrants represent items that lecturers value as either “Important” or 
‘Very important”, and the two bottom quadrants represent items that lecturers rate 
as “Somewhat important” or “Not important at all”. The two left-hand quadrants 
represent items that students participate in at below average frequency. The two 
right-hand quadrants represent items that students participate in at above average 
frequency. If a lecturer deems an item as important or very important and students 
rate it as occurring frequently, it would be shown in the upper right quadrant, 
Quadrant 1. On the other hand, if a lecturer rates an item as somewhat important 
or not important and students report that it happens with low frequency, it would be 
placed in the lower left quadrant, Quadrant 3. If the importance rating of staff and 
the frequency reported by students are aligned, all the CLASSE items would fall into 
Quadrant 1 and Quadrant 3. 
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One of the most valuable ways in which the data can be used is to identify the items 
where students and lecturers are not aligned. Items that occur in Quadrant 4 are 
the ones that lecturers view as being “Unimportant” or “Moderately important” for 
student success, yet the students are reporting that they are participating in these 
behaviours quite often. Therefore, these items could potentially be taking crucial 
time away from more important activities and behaviours. Quadrant 2 shows items 
that are a result of student behaviours that occur with below-average frequency, 
but that the lecturer considers to be important for academic success, and is also 
considered the space of opportunities for improvement. These are the areas where 
the lecturer and his or her students miss each other. By identifying these gaps, the 
data can be used to improve teaching and learning practices in the context of their 
classroom and assist to improving student success rates. 


IN-DEPTH ANALYSIS OF COURSE-LEVEL STUDENT ENGAGEMENT 


To illustrate how CLASSE data can be used to impact the development of effective 
teaching and learning environments, we provide an aggregated case study of four 
universities’ participation in CLASSE. 


CLASSE was piloted in 2013 with a selected group of lecturers across ten courses/ 
modules. In 2014, CLASSE was administered at the University of the Free State 
(UFS) across 15 courses/modules. In 2015, another South African university also 
used the instrument across 18 courses/modules, together with the UFS, bringing the 
total to 48 courses/modules in 2015. During 2016, three South African universities 
participated in the CLASSE, with a total of nine modules. For the purposes of this 
chapter where we aim to reflect on what we have learnt thus far about teaching and 
learning spaces through CLASSE administration, we therefore use data collected 
between 2014 and 2016 (since 2013 was a pilot year). The quadrant analysis 
presented in Table 10.2 is an example of how such analyses are presented to 
participating lecturers. The data presented show an aggregated picture of the 60 
courses/modules represented through 4 089 students and 92 lecturers to allow us 
to determine tendencies. Seven faculties are represented through the 60 modules: 
Economic and Management (31); Education (1); Humanities (11); Engineering (6); 
Law (1); Health Sciences (1); and Natural and Agricultural Sciences (9). The items 
are presented with the count of modules where an item occurred most in brackets. 


As CLASSE is a survey for individual modules, data are not usually combined and 
analysed as a combined data set. Instead, the analysed data, in the form of the 
quadrant analyses of each of the 60 modules, were analysed through frequency 
counts. That is, determining how many times a single questionnaire item appeared 
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in a quadrant. Table 10.3 shows a conditional formatting visualisation to highlight 
most-frequently to least-frequently used techniques. Darker grey areas indicate the 
largest portion of the sample for each activity, and lighter grey areas indicate the 
smallest portion of the sample. Through this approach, the integrity of individualised 
CLASSE data is not compromised. It is also possible to determine which items most 
frequently fall in a specific quadrant. 


TABLE 10.2 Aggregated quadrant analysis for 60 modules 


QUADRANT 2 QUADRANT 1 


Asked questions in class [46] Project requires using various sources [27] 
Participated in class discussions | Attended class prepared [48] 

[42] Worked with classmates on assignments outside of 
Worked with classmates on class [30] 

projects during class [26] Tutored/taught other students [22] 


Combined ideas from different Used email to discuss assignments with classmates [29] 
modules/subjects [24] 


Emailed lecturer [35] 


Discussed grades/assignments 
with the lecturer [42] 


Discussed ideas with the lecturer 
outside class [35] 


Received prompt/informative 


Discussed ideas from class with others [34] 
Perceived the need to work hard [44] 


Received detailed information about learning 
outcomes [54] 


Received motivating interaction from the lecturer [51] 


Work required memorising [31] 
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feedback [34] Work required analysing [57] 

Assignments requiring more than Work required synthesising [56] 

one hour to complete [26] Work required making judgements [43] 

Spent more than 3 hours Work required applying theories and concepts [58] 
preparing for classes [24] Challenging assessment tasks [58] 

Reviewed class notes before Attended class [54] 

class [33] 


Took notes in class [54] 
Studied with classmates [22] 


Easy to follow lectures [34] 


Attended a review session [29] 

Interested in learning course content [60] 
Comfortable talking to the lecturer [51] 
Enjoyed group work [27] 


Challenging course content [48] 
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QUADRANT 3 QUADRANT 4 


Prepared 2 or more drafts of a paper/assignment [26] 
Included diverse perspectives in making points [38] 
Gave a presentation [39] 

Participated in a community-based project [49] 

Wrote papers longer than 5 pages [37] 

Participated in experiential learning [42] 

Participated in service learning [53] 

Participated in fieldwork [45] 

Participated in laboratory work [53] 

Participated in clinical teaching [59] 


Frequency (student rating) 


After combining the analysed data, we can clearly see which items fall in to which of 


Importance (Lecturer rating) 


the four quadrants most often. First, the two quadrants where students and lecturers 
exhibit a degree of similarity (Quadrants 1 and 3), will be explored. The majority 
of the CLASSE items fall in Quadrant 1, where lecturers feel an item is important 
or very important, and students report a high frequency for the item. This indicates 
agreement among lecturers and students about the importance of these behaviours. 
Probably most significant is the high frequency in which lecturers and students agree 
on the importance of interest in learning course content (all 60 modules agree). Also 
showing particularly high frequency in Quadrant 1 are certain behaviours lecturers 
are responsible for, namely, sharing detailed information on learning outcomes; 
providing motivating interaction with students in classrooms; making sure content 
and assessments are challenging; and incorporating analysis, synthesis, application 
of theory, and making judgements into course content. Items showing particularly 
high frequency in Quadrant 1 for which students are primarily responsible include 
attending class and being prepared for class; realising the need to work hard; taking 
notes in class; and feeling comfortable to approach lecturers. The other items in 
Quadrant 1 are not as clear-cut. For example, Quadrant 1 shows that lecturers 
value, and students agree that they enjoy group work (27), however, when looking 
at the conditional formatting visualisation in Table 10.3 and considering where 
frequencies are distributed throughout the other quadrants, 24 modules placed this 
item in Quadrant 2, thereby indicating disagreement between lecturers’ and students’ 
responses. Similarly, while most lecturers place importance on — and most students 
admit to often practicing work requiring memorisation in Quadrant 1 (31) — a large 


amount of students representing 21 modules state that they practice this behaviour 
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even though lecturers do not deem memorisation important (Quadrant 4). Even 
though most lecturers and students agree on the importance of tutoring other students 
(22 modules), a large proportion of responses to this item falls in Quadrant 2 (19), 
as well as seven in Quadrant 3, and 11 in Quadrant 4. This implies that lecturers 
from 18 modules do not regard it important for their students to tutor or teach 


others, even though students from 11 modules engage in such behaviour. 


Quadrant 3 shows items regarded as less or unimportant by lecturers as well as 
less frequent student behaviours. The five items forming part of the Supplementary 
Learning Activities subgroup are all present here. These items include participation 
in service learning, experiential learning, fieldwork, laboratory work, and clinical 
teaching. Since most participating modules do not engage in fieldwork, clinical 
teaching, or laboratory work, it is understandable that these items are not valued 
or practiced by lecturers or students. However, high frequencies in Quadrant 3 for 
participation in community-based projects, service learning, preparing more than 
one draft of an assignment, writing papers longer than five pages, and giving 
presentations might be cause for concern. Teaching others, as well as participating 
in service learning are considered high impact practices (see Chapter 5) for the 
potential of these practices to support the development of valued learning outcomes. 


Quadrants 2 and 4 show where lecturers and students are ‘missing’ each other. 
Quadrant 2 shows several items that lecturers deem important or very important. 
Students report less engagement with these behaviours. This quadrant is therefore 
an important starting point when considering which changes could be made to align 
student behaviours with engagement activities. Some of the items in Quadrant 2 
relate to interactions between students and lecturers. For example, even though 
most students indicated that they are comfortable talking to their lecturers and 
the lecturers deem this as important (Quadrant 1), lecturers would like students 
to engage more in emailing them, and discussing grades and ideas with them. 
Lecturers would also like to see students spend more time preparing for class and 
would like to see more active participation from students in class through having 
them ask questions, participate in discussions, and be able to follow the lectures 
with ease. Lecturers would further like to see more collaborative learning within 
and outside of classes, as well as more synthesis of interdisciplinary thoughts. The 
low participation of students and high value placed by lecturers on receiving/giving 
prompt feedback and doing/giving assignments requiring more than an hour to 
complete act as important prompts for reflection on how lecturers could adapt 
their assessment practices. Quadrant 4, indicating items which are less important 
or unimportant for lecturers, but for which students indicate high participation, do 
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not show any items with the highest frequency. However, students from 20 modules 


indicate that they participate often in work requiring memorisation, even when their 


lecturers do not agree on its importance. 


TABLE 10.3 Conditional formatting visualisation of item-quadrant frequency 


|aı|a2la3 aa | 


2 | Asked questions in class 46 
3 | Participated in class discussions 42 
4 | Prepared 2 or more drafts of a paper/assignment 26 
5 Project requires using various sources 27 
6 | Included diverse perspectives in making points Ea] 38 
7 | Attended class prepared 48 o BE 
8 Worked with classmates on projects during class 26 
9 Worked with classmates on assignments outside 3] 
of class 
10 | Combined ideas from different modules/subjects 
11 | Tutored/taught other students 
12 | Used email to discuss assignments with classmates 
13 Emailed lecturer 
14 | Discussed grades/assignments with the lecturer 
15 Discussed ideas from class with others 
16 | Gave a presentation 
17 | Participated in a community-based project 
18 | Discussed ideas with the lecturer outside class 
19 | Received prompt/informative feedback 
20 | Perceived the need to work hard 
A Received detailed information about learning 
outcomes 
22 | Received motivating interaction from the lecturer 
23 | Work required memorising 
24 | Work required analysing 
25 | Work required synthesising 
26 | Work required making judgements 
27 | Work required applying theories and concepts 
28 | Wrote papers longer than 5 pages 
29 | Challenging assessment tasks 
30 Assignments requiring more than one hour to 
complete 
31 | Spent more than 3 hours preparing for classes 
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Attended class 
33 | Took notes in class 


34 | Reviewed class notes before class 


35 | Studied with classmates 


36 | Attended a review session 


37 | Interested in learning course content 


38 | Comfortable talking to the lecturer 


39 | Enjoyed group work 


40 | Challenging course content 


41 | Easy to follow lectures 


42 | Participated in experiential learning 


43 | Participated in service learning 
44 | Participated in fieldwork 
45 | Participated in laboratory work 


46 | Participated in clinical teaching 


USING CLASSE RESULTS FOR EVIDENCE-BASED EMPOWERMENT 


Once the lecturers receive their module CLASSE reports, they have the benefit of 
empirical evidence of what is happening, or not happening, in their classrooms. 
Based on the results, and especially those presented in Quadrant 2, one can then 
determine what specific improvements will be needed based on what is seen at the 
class level. Staff may be able to evaluate and reflect on their teaching styles and 
adjust them accordingly in order to expose students to activities that can improve 
their learning efficiency (Quimet & Smallwood 2005). It is also a tool for staff to 
see where they have been successful in the motivation, inspiration and influence 
of student learning. Lecturers can also use the information obtained from CLASSE 
to start discussions and debates concerning teaching and learning amongst 
themselves. In essence, students give feedback through the CLASSE that could help 
staff understand their learning processes and influence the practices of teachers. 


Turning back to our quadrant analysis of 60 modules as an example, we share some 
practical next steps. 


Online resources to facilitate evidence-based empowerment 


To facilitate the effective reflection for the lecturer on their CLASSE results or to facilitate 
an academic staff development conversation the SASSE team has development an 
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interactive website. The website links 50 Student Engagement Techniques (SETs) 
developed by Elizabeth Barkley (2010) to the items in the CLASSE. The lecturer can 
click on “Using your Quadrant analysis” on the website https://www. ufs.ac.za/sasse. 
The website connects all relevant SETs to each individual item in the survey. Lecturers 
can select the item that they wish to improve upon, and inspect the relevant SETs 
relating to the item. Once a SET is selected, a summary of the SET is provided, with 
a reference to the full text. In the full reference, the lecturer is provided with step-by- 
step instructions of how to contextualise the technique and how to implement it in 


their classroom. 


It is critical to emphasise that the aim is not to create a mechanistic, decontextualised 
development approach. Rather the website has been developed to create a resource 
that lecturers and academic statf developers can use to have deeper conversations 
on what kind of strategy would work best in the context of their specific classroom 
with their specific students. 


Quadrant 2 in our 60-module example provides several items where Barkley’s 
SETs might provide innovative ways of increasing student behaviours in areas of 
engagement valued by lecturers. For example, if a lecturer wishes to promote 
student participation in class, Barkley proposes eight different exercises ranging from 
introducing visual or physical artefacts as conversation starting point, to having split- 
room debates, where topics are discussed from different viewpoints. 


Towards an evidence-based approach to academic staff development 


Besides focusing on Quadrant 2 to identify techniques to improve engagement 
techniques in classrooms, the CLASSE, by itself or in conjunction with other data 
sources, provide data as starting point for academic staff development conversations. 
For example, from our quadrant analysis example of 60 modules we can identify 
three avenues of conversation. First, using Quadrant 1 to reflect on the quality 
of teaching, learning and assessment happening in class. Similar to the CLASSE 
findings, data from the LSSE and SASSE point to a misalignment between lecturers’ 
low emphasis on memorisation and students thinking it is expected of them (see 
Chapter 4). Conversations about integrating Bloom’s taxonomy, on which these 
items along with memorisation are based, into teaching, learning and assessments 
could impact the quality of students’ learning. 


Second, the items listed in Quadrant 3 serve as reflection about whether lecturers’ 
sense of important engagement activities are consistent with what we know from 
research (e.g. Umbach & Wawrzynski 2005). Our 60-module example shows that 
many lecturers do not consider it important for students to teach others or to present 
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work to others; to write longer papers; to include diverse perspectives when making 
points; or to participate in community and/or service learning. While disciplinary 
differences could to some extent explain why these activities are not considered 
important, the developmental impact of these activities need to be considered 
before dismissing them as unimportant, because in doing so, students also tend to 
dismiss them. 


Finally, items from the quadrant analysis could guide reflection on what is in whose 
power to change. For example, if lecturers want students to participate more in class, 
they have the prerogative to introduce innovative activities to promote this behaviour. 
Similarly, if lecturers want students to engage more with them through email, or 
discussions about grades or other ideas, they could extend their consultation hours, 


explicitly invite students to meet with them, or maybe initiate an email invitation. 


Having reflected on an in-depth analysis of the typical areas in which lecturers and 
student tend to miss each other, we also wanted to reflect on the importance of 
considering the implications of the use of evidence in an academic staff development 


conversation. The reflection is structured using the following guidelines (NSSE 2015): 


1. Make sure lecturers and staff understand and endorse the concept of student engagement 
Staff who are less familiar with assessment in general and the concept of student 
engagement should be convinced of the value of CLASSE results for improving 


teaching and learning. 


2. Understand what CLASSE data represent and use the results wisely 
Understand that CLASSE measures student engagement in a specific class and 
at a specific time. The results cannot be generalised to all the classes which the 
lecturer teaches, but should rather be viewed as an opportunity to encourage 
reflection within a specific context. 


3. Report CLASSE data in a responsible way and make sure staff see the numbers in context 
Use CLASSE data to encourage staff to make improvements. Encourage the 
staff members to use the website or other teaching and learning resources to 
find solutions for their specific context. 


4. Link CLASSE results to other institutional and evaluation data relating to the student 
learning experience 


Student engagement data are most valuable when combined with other 
evidence. These can take the form of module evaluation data or qualitative 


evaluation data of the student learning experience in the module. 
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5. Don’t go at it alone 
Encourage staff to share CLASSE results with each other and to form or join 
conversations of faculty groups or communities focusing on improving teaching 
and learning. The positive impact of CLASSE results will be multiplied if the data 
can be used by groups of staff who are working on improvement efforts around 


campus. 


CONCLUSION 


The chapter introduced the CLASSE and how it can be used at a course/module level 
to promote student engagement and evidence-based improvement in teaching and 
learning. The CLASSE promotes student agency by giving students an opportunity to 
have a say in the quality of their own teaching and learning at the course/module 
level. It allows academics as well as heads of department to monitor the extent 
to which students are experiencing effective educational practices that have been 
implemented in specific courses/ modules. It creates an opportunity for careful 
diagnosis of the climate in the classroom and creates an opportunity to build the 
pedagogical relationship between lecturers and students. 
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STUDENT ENGAGEMENT IN 
SOUTH AFRICAN HIGHER 
EDUCATION: TAKING STOCK 
AND MOVING FORWARD 


Francois Strydom, George Kuh & Sonja Loots 


lt always seems impossible until it’s done — Nelson Mandela 


INTRODUCTION 


Improving student success in higher education is an essential but daunting task. 
The contributors to this book do not underestimate the challenges to overcome, 
yet are cautiously optimistic about making progress in addressing them, and offer 
actionable, material suggestions toward this end. Their analyses are grounded in 
evidence and first-hand experience with developing and implementing promising 
policies and practices at the institution, sector, and national levels. Moreover, their 
reflections are most timely as the stakes have never been higher. While the access 
challenge has not been fully met, notable progress has been made. But access 
without degree completion is a broken promise. And degree completion without 
acquiring the knowledge and proficiencies required to survive and thrive in a 
dynamic global economy is a hollow, meaningless achievement. 


This chapter synthesises the two major student engagement themes — what students 
are doing and how institutions are creating environments conducive to engagement 
— presented in the previous chapters and highlight what we have learnt from the 
data. The chapter closes with recommendations for how to foster greater levels of 
student engagement and success in South African higher education. 


STUDENTS’ EDUCATIONAL BEHAVIOURS 


Arguably one of the most pressing current demands in South African (and international) 
higher education is to understand our students and their learning experiences in 


order to direct efforts towards increasing student success. From an engagement 
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perspective, student success extends beyond passing grades and includes the 
development of cognitive and social-emotional competencies that will prepare 
graduates to apply what they have learnt in the ‘real world’; to make evaluations and 
judgements about information and sources of information (including numerical and 
statistical information); to be able to analyse and synthesise information from various 
sources; to evaluate their own views in relation to others’; and to embrace the variety 
of perspectives usually accompanying diverse people. Our definition of success also 
includes acquisition of other proficiencies that speak to the demands placed on 
21st century graduates by employers. Amongst others, these include being able to 
actively participate in a team; being able to work with and learn from diverse people 
and environments; and the ability to recognise and live out the social responsibility 


of a democratic citizen. 


The sections that follow describe in which engaging activities students actively 
participate. We explore the kinds of academic challenges they are presented with, 
how they are learning with their peers, and the various effective educational practices 
they experience. 


Academic challenge 


Taken together, the chapters in this book offer a fair amount of insight into students’ 
development of cognitive skills as represented through their scores on the Academic 
Challenge scales of Higher-Order Learning; Reflective and Integrative Learning; 
Quantitative Reasoning; and Learning Strategies. 


First, the South African Survey of Student Engagement (SASSE) results presented 
in Chapter 2 show that students report a relatively high (mean of 40 out of 60) 
development of Higher-Order Learning consisting of the ability to analyse, apply, 
evaluate, and synthesise information through their studies. When looking at how 
these cognitive skills are developed, students report higher mean scores for Higher- 
Order Learning when they feel challenged by their subject material, when they spend 
more time reading and preparing for class, and when they are assigned more writing 
tasks. Chapter 9 also shows that when students prepare for class they perform better 
academically, as one might expect. That said, a summary of 60 modules in which 
the Classroom Survey of Student Engagement (CLASSE) was administered shows that 
these lecturers do not deem writing assignments longer than five pages as important, 
nor do the students indicate that they are given many such assignments (Chapter 10). 


Second, even though there is a slight increase from first-year to senior year students’ 
mean scores on the Reflective and Integrative Learning indicator, the participation 
in these activities remains low (mean of 33 out of 60). The behaviours making 
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up this scale include connecting learning to societal problems or issues; including 
diverse perspectives in discussions or writing assignments; examining strengths and 
weaknesses of own views; seeing things from another’s point of view; connecting 
ideas from modules to prior experiences; and learning something which has changed 
the way they understood issues or concepts. The analysis of one university sample in 
Chapter 9 showed that none of the items in the Reflective and Integrative Learning 
scale impacted on students’ average marks. Further, Chapter 2 reported that less 
than half of the students connect what they learn to societal issues. 


The final take away from the data about Academic Challenge, is the relatively low 
expectations students have for developing quantitative reasoning, or using these 
competencies (Chapters 2 and 3). The Quantitative Reasoning indicator captures 
whether students are able to draw conclusions based on the analysis of numerical 
information; evaluate others’ conclusions of numerical information; and translate 
numerical information to real-world issues. In an increasingly data-driven world, 
where information from different sources (some more reliable than others) are 
instantly available via various technological means, it is essential to be able to make 
informed judgements about this information. The low mean scores for both students’ 
expectations and application could be interpreted bearing in mind the general lack 
of connecting what they are learning to the real world as indicated earlier. In other 
words, although the chances are good that they are practicing quantitative reasoning 
in their daily lives, students might not make the connection between what they are 
doing and the items in a survey administered in the context of higher education. 


Learning with peers 


The two indicators relevant to the Learning with Peers cluster of competence include 
Collaborative Learning and Discussion with Diverse Others. Regarding Collaborative 
Learning, Chapter 3 showed that students entering university do not expect to 
collaborate much with other students. However, the SASSE shows an increase from 
their expected engagement in Collaborative Learning to their actual engagement 
in Collaborative Learning and even more so as students progress from their first to 
senior undergraduate years (Chapter 2). Chapter 5 also shows that 73% of students 
report being asked to participate in group work. While these findings look promising, 
there are still some improvements to be made. For example, data from the CLASSE 
(Chapter 10) tell us that many of the lecturers from the 60 modules would like to see 
more in-class collaborations between students; at the same time, about a third of the 
lecturers express the view that it is not important for students to work with classmates 


on assignments during class. 
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Upon starting at university, many students expect to interact with diverse others. 
In fact, the SASSE results in Chapter 2 show that over 60% of students indicate 
that they have regular interactions with people who are different from themselves in 
terms of socio-economic status, race, religion, or political views. These students also 
report more positive interactions with other students and staff and find the campus 
environment more supportive. Chapter 9 also shows that students who engage with 
others who differ from them in terms of socio-economic status and race, perform 
better academically. 


Effective academic behaviours 

The data reported in Chapter 9 show us that some behaviours have a direct positive 
impact on students’ academic performance. These include asking questions in 
class; preparing for class; talking to lecturers about career plans; explaining subject 
material to others; perceiving the academic content as challenging; and engaging 
with diverse others. Finding ways to build on these behaviours, as well as developing 
other effective academic behaviours to impact students’ pathways to success are 
pivotal focal points to enhance engagement. 


In sum, the chapters in this book have identified several effective behaviours students 
are doing which are increasing their engagement and helping them navigate 
successfully through higher education. These include: 

= preparing for class; 

= reading and writing academic material; 

= feeling challenged; 

= participating in class discussions and asking questions; 

= talking to lecturers about careers; 

= participating in practical work and group work; 

= increasing collaborative work with peers in time; and 


= interacting with diverse others. 


ENVIRONMENTS CONDUCIVE TO ENGAGEMENT 


While students’ behaviours, agency and motivation are important contributors 
to their levels of engagement, creating an educational environment that enables 
students to actively engage in forms the foundation of student success. Three areas 
of environmental support highlighted throughout the book seem to be especially 
important: students’ views on how supportive their educational environments are; 
the relationship between students and lecturers; and classroom environments. 


222 


STUDENT ENGAGEMENT IN SOUTH AFRICAN HIGHER EDUCATION: TAKING STOCK AND MOVING FORWARD 


Supportive campuses 


The information presented in Chapter 2 showed that students generally report mixed 
views as to whether their campus environments are supportive of their success. Their 
general perception of the quality of interactions with other students, academic staff 
and non-academic staff seem to be of good quality. However, while most students 
feel satistied with the quality of their interactions with other students and academic 
staff, only half indicated that the quality of interactions with administrative and/or 


support staff were good. 


Students who reside on campus report slightly higher quality of interactions with 
student support services, and experience notably more institutional emphasis on 
students’ well-being, attending enriching events, studying and academic work, as well 
as providing academic support (Chapter 7). In fact, living on campus not only makes 
a difference in how supportive students perceive the university to be, it also is positively 
related to the frequency with which students interact with diverse others as well as the 
frequency of interactions with lecturers (Chapter 2). Compared with seniors, first-year 
students perceive the campus as more supportive (a finding that mirrors national data 
from the United States), more frequently apply learning strategies, and are exposed 
more often to effective teaching practices. In Chapter 7 one area that needs attention 
is pointed out: while most students feel that their institutions place a lot of emphasis 
on studying and academic work, they do report that the institutions are not providing 
enough support for them to perform well academically. Another area where most 
student responses signal a potential challenge is relatively low scores on whether their 


institutions place emphasis on providing support for their overall well-being. 


Interactions between students and academic staff 


When students enter higher education, they do not expect to spend much time 
interacting with lecturers (Chapter 3 reports a mean of 30 out of 60). These 
interactions refer to talking about their career plans, academic performance, or 
discussing ideas outside of class with lecturers, as well as working with academic 
staff members on projects beyond academic work. Even more worrisome is that 
the majority of students answer “Never” or “Sometimes” when asked whether they 
actually do interact with academic staff in these ways (Chapter 2). The positive 
effects of student-lecturer contact has been well documented internationally (e.g. 
Kuh, Kinzie, Schuh & Whitt 2010; Mayhew et al 2016); in fact, the SASSE findings 
reported in Chapter 9 show the positive relationship between students’ academic 


performance and their discussions with lecturers about their career plans. 
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Adding to the complexity of student-staff relationships is a lack of alignment and 
understanding between both parties. For example, when comparing the Lecturer 
Survey of Student Engagement (LSSE) analysis with SASSE findings, lecturers seem to 
have an unrealistic view of students’ life demands, for example, expecting students 
to spend a significant amount of time preparing for class and notably overestimating 
the amount of time students spend socialising or relaxing. That said, the emphasis 
lecturers place on class preparation might have some salutary effects, considering 
the positive relationship this behaviour has with academic performance (Chapter 9). 
Lecturers, academic support staff, or peer facilitators should, therefore, help students 
to apply more effective and time-efficient strategies to optimise class preparation. 


Effective teaching practices 


Teaching and learning in classrooms, laboratories, and studios continue to be the 
venues in which most students are likely to be academically challenged. Chapter 9 
pointed to a positive relationship between students’ academic performance and 
effective teaching practices, including the provision of detailed feedback shortly after 
assessments; when module outcomes and requirements are clearly explained; and 
when module content is presented in an organised way. For some, these practices 
might seem obvious and foundational as classroom practices. Even so, SASSE results 
show that more than 20% of students indicate that their lecturers do not clearly 
explain subject/module outcomes and requirements; about one fifth (19%) report 
that their lecturers do not regularly present sessions in an organised way, and a third 
(34%) say they do not get detailed feedback in a timely manner. 


It is somewhat surprising that SASSE data did not find statistically significant 
relationships between students’ academic performance and making class 
presentations or working with others on projects or assignments. Nor was the 
Reflective and Integrative Learning scale positively related to academic performance 
as measured by students’ average marks. Perhaps this is due in part to few such 
practices being used in classrooms. For example, the CLASSE analysis shows that 
student presentations are not valued by lecturers nor do students get much practice 
in making presentations. This points to the absence of alignment between pedagogy 
and desired outcomes such as deep learning as discussed in Chapters 2, 4 and 9. 


Also, as explained in Chapter 4, academic staff still primarily use lecturing, with 
minimal inclusion of group discussions as teaching methods. Experiential learning 
opportunities, including giving presentations, independent student work, and showing 
movies/videos rarely or never happen according to the LSSE respondents. Given 
that lecturers indicate they spend an average of five hours per week on working 
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to improve their teaching (which includes self-reflection, meeting with teaching 
consultants, attending teaching workshops, and conducting research on their own 
modules/subjects), it is worrisome that the primary mode of delivery is still traditional 
lecturing. The CLASSE analysis in Chapter 10 also tell us that lecturers would like 
to see students participate more in class discussions, ask more questions and work 
more with others during class time. However, if the teaching strategies do not change 
to create environments more conducive to such engagement, it is difficult to see how 
it can be expected of students to engage and participate more. 


Students from 20 of the 60 modules in which the CLASSE was administered indicated 
that their modules emphasised memorisation, even though lecturers reported they did 
not place much value on memorisation (Chapter 10). This finding is corroborated by 
the information in Chapter 4, where a comparison of SASSE and LSSE findings show 
a mismatch between lecturer and student perspectives on the emphasis placed on 
memorisation. More experienced lecturers also rely less on memorisation and more 
on developing other cognitive skills; however, these lecturers are also less inclined 
to adopt new teaching methods. 


Consider, too, that almost half of the lecturers responding to LSSE do not think it 
is important for students to include diverse perspectives in module discussions. In 
addition, more than a third do not think it is important for students to develop an 
international perspective or to conduct research with lecturers, or to participate in 
service learning or community projects (Chapter 4). The findings summarised in 
Chapters 5 and 10 corroborate these patterns. With the current debates on the 
decolonisation of higher education curricula, teaching, and institutional culture in 
general emphasising the value of including diverse perspectives has never been 
more important to fostering a civil society and equipping students to work effectively 
with people from backgrounds different than their own. 


Taken together, these findings highlight the importance of efforts underway to improve 
the teaching and learning skills of academics, perhaps in the form of postgraduate 
diplomas or continuous professional development workshops. 


In sum, the chapters in this book highlight the following environmental factors that 
make a positive impact on students’ engagement: 


= Even though the student-staff interaction indicator score is quite low, relationships 
between students and lecturers seem good. Additional inquiries into how to use 
positive relations between students and staff are warranted. 


= Living on campus is related to mixed influences on student engagement. It is 
therefore important to further explore what the contributing factors are to this 
group’s engagement and to find ways of expanding these factors to other students. 
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= As expected, engaging pedagogical approaches are positively related to students’ 
academic performance. 


However, from the student perspective, the university environment could do more to 
address their academic and social needs. While students perceive they are expected 
to perform well academically, the data do not show that students feel connected to 
their lecturers beyond the classroom. Further, the quality of relationships with non- 
academic staff is rated quite poorly, which could impact whether or where students 
go to find help. 


Particular contextual challenges to consider 


The data from this book encourage reflection on some specific contextual 
challenges, including the persistent racial and generational gaps in how students 
are experiencing financial stress; supporting first-year students’ transition from high 
school to university as well as developing their learning from first to subsequent 
years of study; and developing academic staff as well as nurturing the student-staff 
relationship. We will touch on each of these briefly. 


From what we know from the Financial Stress Scale, most students — particularly black 
African and first-generation students — have a persistent struggle with making ends 
meet financially. For example, the scale shows that black African — and particularly 
first-generation black African students from both genders — worry most about paying 
university fees and have been affected most in terms of food insecurity. In fact, 
first-generation students seem to be most affected even within racial groups. For 
example, there is about a 20% difference between black African first- and non-first- 
generation students when students indicate that they have never or sometimes run 
out of food without being able to buy more. Yet, even when faced with financial 
stress, and in many cases hunger, only half of students would consider dropping out 
because of financially-related reasons. 


Regarding developmental growth from first year to senior undergraduate years, the 
BUSSE data confirm the so-called “freshman myth” phenomenon, where students 
overestimate their abilities and preparedness and underestimate the challenges 
higher education presents. However, some expectations students have as they start 
their higher education studies turn out to be realistic. For example, students generally 
do not expect to engage often in collaborative learning activities, frequently interact 
with staff or have serious discussions with peers from backgrounds different from 
their own. The SASSE data collected later in the same academic year confirm that 
students behave consistent with these expectations. In general, first-year students 
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perceive institutional environments to be more welcoming, and as mentioned earlier, 
use learning strategies more often, and experience more effective teaching practices. 
An optimistic interpretation of this pattern of results may in part be a function of 
the widespread movement by institutions to increase transitional support for first- 
year students. 


Participating in deep learning activities enables students to understand the purpose, 
meaning, and significance of what they are learning and requires an active 
learning process involving the generation of ideas, pattern recognition, and making 
evaluations about the logic of the argument (Entwistle 2000). Only slight variations 
were documented in the development of a capacity for deep learning between the 
first and later years of study, and no differences in students’ ability to evaluate and 
create knowledge. Indeed, the results point to an overemphasis on memorisation, 
contributing to surface learning which is orthogonal to deep learning. 


The LSSE analysis shows that lecturers consider the development of students’ 
cognitive skills to be only somewhat important. This suggests the planning of module 
and course outcomes, assessment, and consecutive year curricula within disciplines 
warrant attention to efforts that can improve cognitive development and the 
associated deep learning competencies. This should include, for example, reducing 
the amount of lecture time and increasing the use of more engaging teaching and 
learning approaches. Another factor undermining the development of these skills is 
the disconnect between lecturers and students on how much emphasis is placed on 
memorisation within their modules as noted earlier. According to Mayhew et al the 
importance of moving away from the dominant lecture format to more frequent use 
of engaging pedagogies is underscored in the most recent synthesis of research 
about what matters to student learning in higher education: 

Seemingly irrefutable evidence demonstrates that active and engaged 

learning practices yield substantial benefits over traditional lecture- 

based formats in which students passively receive information. These 

outcomes include greater verbal and quantitative skills, subject matter 

competence, cognitive and intellectual skills, openness to diversity, 


intercultural competence, leadership, citizenship, and moral development 
(Mayhew et al 2016:550). 


That some lecturers do not value presenting diverse or international perspectives in 
their modules is also worrisome, particularly considering the benefits that accrue to 
students who interact with diverse others. Recognition and appreciation of diversity 
— whether encountered through curricula, teaching and/or learning methods, 
assessment, or personal interactions — are fundamental aspects of higher-order and 
reflective learning. How else can students learn to make evaluations or judgements 
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if not given different types of information; how will they learn to respect viewpoints 
if not confronted with different opinions; or how will they learn to form their own 
reasoned opinions if not challenged by others’? 


Another finding for academic staff to reflect on is the type of relationship they aim 
to have with students. The items measuring student-staff interaction all refer to 
discussions about personal development (e.g. career plans, academic performance, 
or working with staff on other projects), or co-curricular conversations on subject- 
related topics. If, as the students indicated, they are comfortable talking to their 
lecturers and the lecturers would like more such interactions with their students, 
as they indicated, more could be done to help create student-staff relationships, 
which could impact how students navigate through their studies. Building on 
this relationship is crucial for the developmental potential such interactions with 
lecturers might hold for students, but also because interactions with lecturers could 
act as gateways to discovering more about institutional and disciplinary cultures 
and resources. As Mahat and Goedegebuure (2014) stated, students are the least 
informed of all stakeholders in terms of higher education choices, outcomes and 
processes. Enhancing the relations between students and staff might therefore also 
have the salutary effect of making students more valued by their institution. 


THE WAY FORWARD 


The common theme throughout all the chapters in this book is that student 
engagement is an internationally recognised construct and set of student and 
institutional behaviours associated with student success. The contextualisation of 
engagement measures to different countries is one of the ways in which we are 
coming to understand undergraduate student experiences of diverse populations 
across the world. In addition, understanding student experiences and performance 
has become increasingly important to restore public confidence in higher education 
(Kuh et al 2015). This is especially true in South Africa where the value and purpose 
of higher education is being considered in terms of its funding, its quality, and 
relevance to the post-colonial African context. 


The analyses offered in this book provide evidence that focusing on ways to 
increase student engagement can make a material positive difference to students’ 
chances of success. The following themes summarise some of the main challenges 
and opportunities that lie ahead synthesised from the information reported in the 
various chapters. 
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Developing a systemic understanding of the student experience 


More intersectional analyses are needed to get a more granular understanding of how 
different student populations experience higher education. The SASSE project now 
involves 19 higher education institutions, 15 of which are public institutions, three 
private, and one international institution. Thus, now is a propitious time to further 
expand and support a community of practice that develops and tests interventions 
based on engagement data. Keys to such an effective community of practice are 
developing a common engagement language and finding ways of making student 
engagement data consequential so that it changes institutional policy and practice. 


\" 


By “consequential” we mean gathering data that answer important questions about 
student and institutional performance and turning that information into evidence 
that informs and guides improvement efforts. Another characteristic of consequential 
assessment is communicating the evidence in meaningful ways to various audiences 


including what the institution is doing to enrich student accomplishment. 


Understanding and supporting students entering higher education 


The introduction of the Financial Stress Scale allows institutions to understand the 
challenging, non-negotiable financial pressures that severely limit students’ chances 
to survive and thrive during their higher education studies. Three promising areas of 
research with the BUSSE are: (1) the use of the data in academic advising; (2) early 
warning systems; and (3) predictive analytics. The University of the Free State (UFS) 
SASSE team has generated contextualised advising reports which provide information 
on themes considered important to advancing student success. These faculty reports, 
generated by using BUSSE data, provide faculties with profiles of incoming cohorts 
and assist to address the self-reported transitional difficulties identified by students. 
However, this project is still at an early development stage. Predictive analytics 
is a key focus of universities, such as Georgia State University (2016), that have 
successfully reduced the achievement gap between different groups of students, 
especially those from historically underserved backgrounds. The use of engagement 
data in predictive analytics is at an early stage. The Siyaphumelela programme, 
funded by the Kresge Foundation, supports five South African institutions to improve 
their institutional capacity to collect and analyse student data (including student 
engagement results) and to promote the use of data analytics to improve student 
success (Siyaphumelela n.d.). The ethical and professional use of this information is 
currently being considered. 


229 


CHAPTER 11 * STRYDOM, KUH & LOOTS 


Building academic capacity through student engagement 


Chapter 4 offered some ideas of how LSSE data could help inform interventions 
springing from a policy initiative such as the University Capacity Development 
Grant. The use of LSSE data in systemic and institutional staff development needs 
to be explored in a more intentional way across institutions. Based on the data 
shared, innovative methods to develop and support academic staff need to be 
implemented. This particularly relates to educating lecturers on ‘what works’ in terms 
of teaching methods and strategies. It also implies sensitising lecturers to possible 
misconceptions they and their students might have about each other, as well as 
learning and relationship expectations. To do this essential work well will require 
adequate technological and academic support to create sustainable learning 


environments conducive to engagement. 


Developing South African high-impact practices and pedagogy 


The initial analysis of students’ participation in high-impact practices highlighted 
the need for cross-institutional analysis to assess which of the proposed potential 
high-impact practices will be effective in the South African context. The identification 
of high-impact pedagogical practices (Chapter 9) will require a similar cross- 
institutional analysis to see if there are classroom practices that can make a difference 
to student success. 


Furthering evidence-based development and change 


The analysis and preliminary success with the use of the CLASSE suggest that combined 
with the development of innovative online support, the use of student engagement 
evidence could be enhanced. Making evidence about student engagement 
consequential is essential to cultivate a culture of institutional ownership of assuring 
quality, as contrasted with the compliance mentality that tends to dominate when 
responding to accountability requests and publicly sharing information about student 
and institutional performance. In a culture of ownership, evidence about students’ 
experience and institutional performance is interpreted, integrated and reflected on 
to facilitate a holistic understanding from which to formulate implications for action 


(Kuh et al 2015). 


Designing and organising for student success 


Edward Deming is often credited with asserting something akin to the following: 
“every system is perfectly designed to get the results it gets”. This observation raises 
the question of how higher education institutions can be re-engineered to produce 
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different, better results. We posit that five design and organising principles adapted 
from Kuh et al (2010) are indispensable to realise the needed systemic changes 
required to advance the student engagement and success agenda in South African 
higher education. Each of these interventions must be adapted and implemented 
with fidelity to fit local contextual features. 


Principle 1: Feature student success in the institution's ‘living’ differentiated mission 
and its lived educational philosophy 


For South African higher education to produce different results, the types of institutions 
and their missions need to be differentiated in order to provide a guiding vision for 
the staff in these institutions. A focus on student learning and success should be 
clarified and translated into plain language for all academic and support staff as 
well as students so that it is lived out throughout the institution and informs everyday 
practice. To ensure that this mission is enacted, senior leadership must publicly 
champion the importance of student success. The development of this mission must 
be underpinned by an education philosophy that reflects the importance of curriculum 
contextualisation and pedagogical innovation appropriate for the institutional and 
national context. In addition, as recommended in Chapter 6, allocating specific 
roles to relevant people will add structure to the pursuit of engagement and student 
success, while also keeping staff from feeling overwhelmed. 


Principle 2: Make talent development the central tenant of the institution’s operational 
philosophy and sector mission 


Universities are the engine rooms of the knowledge economy across the globe 
because they develop talent (Astin & Antonio 2012; Merisotis 2015). Talent can only 
be developed if high expectations are set for everyone in the institution — students as 
well as academic and support staff. 


Firstly, the process of talent development starts with knowing your students. Information 
presented in Chapters 2 and 5 suggest that the assessment of higher-order skills 
needs to be expanded in institutions and that high-impact practices need to be 
introduced to help graduates acquire the attributes needed to grow the economy 
and allow individuals to participate fully in the growing economy. The performance 
standards we set for our students need to be high but attainable and consistent with 
their preparation levels. 


Secondly, higher expectations of our students will require more investment in 
innovatively planning to develop the talent of staff — their pedagogical approaches 
and related skills — to create appropriate quality support that optimise students’ 
chances of attaining a quality education. South African higher education 
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institutions are going to have to find innovative ways of bringing staff and students 
together, especially after the student protests, which indicated the need for greater 
mutual understanding. 


Thirdly, human resource practices need to focus on attracting, socialising and 
rewarding competent people. This means recruiting academic and support staff 
who are committed to student learning and success, and aligning the reward 
system with the student success mission, values and priorities of the institution. 
Institutional leaders must be appointed who are right for the times, campus culture 
and institutional trajectory. Student-centred policies and practices must be central to 
academic and support services. A focus on talent development requires investments 
in teaching and learning centres and activities that contribute to student success such 
as opportunities to apply their learning (service learning, internships, undergraduate 
research projects, etc.). Budgeting models and processes that privilege student 
learning processes are also needed. 


Principle 3: Create learning environments that value and promote inclusion and 
educational enrichment through equity-minded policies, programmes, and practices 


Cultivating an inclusive equity-minded institutional culture that embraces and 
promotes diversity in all its forms is foundational to engage all students at high levels 
and support their learning and success. The physical plant aspects of a learning 
environment must be addressed by designing multi-functional learning spaces 
that allow for quick changes in furniture arrangements during classes to facilitate 
active and collaborative learning. Modifying library spaces to accommodate a wide 
range of learning preferences from formal to informal, or face-to-face to online 
learning. The curriculum design must encourage and support the full participation 
of students with disabilities and students with varying levels of preparedness. The 
focus on decolonisation highlights the need for the curriculum content that includes 
a diversity of voices and incorporates high-impact learning experiences that illustrate 
the relevance of learning for contemporary challenges. An inclusive culture also 
requires the development of a prominent educational and support role for student 
affairs to help foster co-curricular environments that promote inclusion, cohesion 
and student success. 


Principle 4: Cultivate an ethic of continuous improvement and collaboration 
Educationally effective institutions are characterised by positive restlessness (Kuh ef al 
2010). They sense that at any given moment one or more aspects of student and 
institutional performance can be improved. This animating impulse is not born out 
of institutional inferiority or insecurity, but rather confidence and commitment to high 
performance. 
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Consistent with this principle, the Quality Enhancement Project of the Council on 
Higher Education with its appreciative inquiry approach is a systems level example 
of how institutions can build on and develop confidence in their initiatives by using 
evidence to continuously improve and contextualise interventions to be effective and 
efficient. By integrating student engagement data into monitoring and evaluation 
frameworks, institutions can monitor the quality of the student experience. A critical 
component of positive relentlessness or an improvement-orientated culture is 
recognising the need to ‘sunset’ less effective programmes and activities to support 
high-priority initiatives. Improvement-orientated institutions encourage and reward 
cross-functional activities focused on student success and tighten the philosophical 
and operational linkages between academic and student affairs as suggested in 
Chapter 7. This collaboration enables them to harness expertise and resources, 
especially in resource constrained circumstances. These institutions are also 


responsive to the needs of their surrounding communities. 


Principle 5: Clarifying the pathways that maximise student success 

Institutions focused on promoting student success provide alternative unfettered 
pathways that students can follow to reach their educational objectives, and 
make students aware of these pathways through intrusive academic advising and 
associated practices. This will require frontloading resources to smooth the transition 
from school to higher education and also for initiatives that help smooth and 
lend coherence to transitions between different years of study and also between 
institutions. South African higher education has made some strides in developing 
first-year experience models that have been shown to provide students with clearer 
communication regarding what it means to be successful. The authors of this book 
highlight that more needs to be done to provide support to senior students through 
improved pedagogical approaches as well as the integration and scaling of high- 
impact practices in undergraduate education programmes. Institutions also need 
to consider the level and type of support needed by students facing additional 
challenges, such as students with disabilities, first-generation students, or students 
entering higher education from poor socio-economic circumstances. Student 
engagement data from SASSE and its sibling surveys can be used to develop a more 
nuanced, evidence-based approach to orientation and transition programmes for 
first-year and senior students. New developments in the field of predictive analytics 
and academic advising and coaching are promising interventions to address the 
challenges facing students and institutions. 
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LAST WORDS 


This book started out by asking: What would a transformed higher education system 
look like? What is the contribution of higher education to a country? How do we 
create institutions that are ready to welcome and educate a wide range of students 
from diverse backgrounds? And how do we create institutions that challenge our 
students to develop the next big idea and address 21st century challenges while 
providing them with the support that enhances their chances of success? While the 
aim of the book was not to provide answers to these grand tour, philosophical 
questions, we are confident that student engagement is a central catalyst to this 
work and to students’ pathways to success. Further, through providing a birds- 
eye view of what is happening in higher education through synthesising data from 
various engagement information sources, this book lays out some ways to shape 
an evidence-based culture of decision-making within higher education. Equally 
important, the data featured in this volume provide a foundation for much needed 
reflection and discussions on the questions raised above as well as about more 
practical approaches to enhance engagement and student success. 


The last remaining question is whether we have the will to do so. The only acceptable 


answer is, we must. 
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Engaging Students is the first comprehensive manuscript on research 
into student engagement in South African higher education. Each 
of the chapters has a clear, distinguishable focus and makes an 
original contribution in and of itself, while retaining the central 
focus on student engagement. The book provides a rich, informative 
picture of the current state of student engagement evaluation, while 
also highlighting challenges and opportunities for future advances. 
A particular strength of this publication is its emphasis on the 
importance of taking evidence-based decisions, and showing how 
the South African Survey of Student Engagement (SASSE) can provide 
the evidence for well-informed changes in policy and practice in 
order to enhance student success. 
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